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Abstract 


The  purpose  of  this  study  was  to  examine  the 
attitudes  of  senior  level  managers  in  Air  Force  System 
Command's  Aeronautical  Systems  Division  regarding  the 
utility  of  the  Air  Force  R&M  2000  Program. 

Five  investigative  questions  were  asked:  1)  What  is 
the  effectiveness  of  existing  R&M  management  tools  as  aids 
to  performing  functional  management  duties?  2)  How  are  the 
ten  integrated  logistics  support  elements  influenced  by 
increased  emphasis  on  R&M?  3)  What  priority  is  given  to  R&M 
2000  goals  by  senior  level  ASD  managers?  4)  How  satisfied 
are  senior  level  ASD  managers  with  R&M  education  and 
training,  and  R&M  expertise  within  their  program  offices? 
And,  5)  How  effective  is  the  R&M  2000  Program  within  ASD 
program  offices,  and  how  can  this  effectiveness  be  improved? 

A  survey  was  used  to  collect  the  research  data. 

Findings  have  determined  that  the  R&M  management  tools 
presented  in  the  survey  were  moderately  effective  as  aids  to 
planning,  organizing,  directing,  and  controlling  R&M 
activities.  Additionally,  program-specific  R&M  requirements 
had  a  moderate  influence  on  the  management  of  integrated 
logistics  support.  ASD  senior  managers  disagreed  with  the 
priority  order  given  the  goals  of  R&M  2000  by  HQ  USAF. 

Also,  they  are  not  fully  satisfied  with  the  methods  of  R&M 
education  and  training  utilized,  nor  are  they  fully 
satisfied  with  the  amount  of  R&M  expertise  resident  in  ASD 


program  offices.  Finally,  ASD  senior  level  managers  felt 
the  Air  Force  R&M  2000  program  was  moderately  effective 
within  ASD  program  offices. 

Deficiencies  were  identified  and  recommendations  made 
for  improvement  of  the  design  of  the  survey  instrument  prior 
to  its  reuse  for  future  research.  Also,  it  was  recommended 
that  other  AFSC  product  divisions  have  the  same  survey 
administered  to  them  for  comparison  of  results. 

Appendices  include  a  copy  of  the  research  instrument 
and  its  cover  letter  (Appendix  A),  verbal  responses  to 
applicable  survey  questions  (Appendix  B),  additional  data 
analysis  support  materials  (Appendix  C),  and  a  comprehensive 
list  of  abbreviations  found  throughout  this  document 
(Appendix  D). 
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THE  R&M  2000  PROCESS  AND  RELIABILITY 
AND  MAINTAINABILITY  MANAGEMENT: 

ATTITUDES  OF  SENIOR  LEVEL  MANAGERS 
IN  AERONAUTICAL  SYSTEMS  DIVISION 

I.  INTRODUCTION 

Overview 

In  December  1985,  the  Secretary  of  the  Air  Force 
distributed  a  Fact  Sheet  explaining  the  new  Air  Force 
Reliability  and  Maintainability  (R&M  2000)  Program.  In  this 
fact  sheet,  explicitly  stated,  were  the  goals  and  objectives 
of  this  new  "corporate  acquisition  philosophy."  Its  intent 
was  to  outline  a  new  way  of  orienting  our  thinking  about 
priorities  within  the  acquisition  world.  Soon  to  follow 
were  changes  to  AFR  800-18.  Air  Force  Reliability  and 
Maintainability  Policy;  DoDD  5000.3,  DoD  policies  and 
procedures  for  operational  test  and  evaluation  during  weapon 
system  acquisition;  and  DoDD  5000.4,  which  establishes 
policies  and  responsibilities  for  R&M  for  major  weapon 
system  acquisition  and  life  cycle  cost  management.  In 
addition,  numerous  (five,  to  date)  R&M  policy  letters  have 
come  from  the  Office  of  the  (Air  Force)  Chief  of  Staff 
providing  guidance,  measurement  standards,  and  personnel 
performance  measureme.it  criteria  ( 19;27;28;29;30) . 


In  his  recently  published  book  The  Defense  Matrix : 
National  Preparedness  and  the  Military  Industrial  Complex , 
retired  General  James  P.  Mullins,  former  Commander  of  Air 
Force  Logistics  Command,  faults  the  military  for  its  long¬ 
standing  "rubber  on  the  ramp"  mentality.  This,  he  says,  has 
resulted  in  our  acquiring  military  systems  without 
sufficient  regard  for  the  means  to  insure  proper  operation 
and  support.  General  Mullins  feels  Air  Force  acquisition 
leaders  have  established  a  mindset  of  "institutionalized 
mediocrity"  or  the  ready  acceptance  of  unreliable  equipment. 
He  feels  the  technology  is  available  to  give  us  far  greater 
weapon  system  reliability,  if  acquisition  managers  only  ask 
for  it  (7:109).  Not  without  motivation  for  concern,  this 
move  towards  focusing  thinking  on  issues  of  R&M  is  echoed  in 
three  current  studies  (22;32;33)  and  numerous  publicly 
acknowledged  acquisition  errors — the  Sergeant  York  Gun 
(4;15;21),  the  much  maligned  B-l  bomber  (2;13;16;20;35) ,  and 
the  Advanced  Cruise  Missile  (13;31),  to  name  a  few. 
Ultimately,  it  is  desired  that  thinking  "R&M  2000*  will 
instill  in  the  end  user,  acquisition,  and  support 
communities  an  attitude  which  places  R&M  considerations  on 
par  with  cost,  schedule,  and  performance  (37:4). 

Problem  Statement 

No  rigorous  statistical  study  of  management  opinion  and 
attitude  has  been  accomplished  since  the  institution  of  R&M 
2000  as  a  management  philosophy.  Hence,  no  one  knows  if  R&M 
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2000  has  truly  been  accepted  by  the  people  charged  with  its 
adoption. 

Justification 

On  1  October  1986,  HQ  USAF  directed  the  Air  Force 

Institute  of  Technology  (AFIT)  to  maintain  an  Air  Force 

Center  of  Excellence  for  Reliability'  and  Maintainability 

(CERM).  The  Center's  charter  is 

to  develop  R&M  concepts,  theory,  and  techniques 
and  to  provide  R&M  consulting  services  to  Air 
Force  organizations.  (10:7) 

This  was  one  of  the  first  research  projects  to 
investigate  the  impact  of  R&M  2000  on  the  management  of 
acquisition  programs  initiated  under  this  tasking.  It  is 
responding  to  senior  Air  Force  leadership's  desire  to  keep 
informed  on  how  the  message  of  increased  emphasis  on  R&M  is 
being  received,  accepted,  and  implemented  in  the  field. 

Research  Question 

What  are  the  attitudes  of  senior  level  managers  in 
Aeronautical  Systems  Division  (ASD)  regarding  the  utility  of 
the  Air  Force  R&M  2000  Program? 

Investigative  Questions 

The  questions  to  be  investigated  in  this  research  are: 
1.  How  do  senior  ASD  managers  feel  about  the 
effectiveness  of  existing  R&M  management  tools  as  aids  to 
performing  functional  duties? 
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2.  How  are  the  ten  integrated  logistics  support 
elements  affected  by  increased  emphasis  on  R&M? 

3.  What  priority  is  given  to  R&M  2000  goals  by 
senior  level  ASD  managers? 

4.  How  satisfied  are  senior  level  ASD  managers 
with  R&M  education  and  training,  and  R&M  expertise  within 
their  program  offices? 

5.  How  effective  is  the  Air  Force  R&M  2000 
program  within  ASD  program  offices,  and  how  can  this 
effectiveness  be  improved? 

Research  Propositions  and  Hypotheses 

This  study  addresses  the  five  research  objectives 
through  three  propositions  and  two  independent  hypotheses 
which  directly  support  two  of  the  research  objectives.  The 
proposition-null  hypothesis  relationships  investigated  in 
this  study  are: 

Proposition  1^.  The  tools  of  R&M  management  addressed 
in  this  research  study  have  had  little  or  no  effect  on 
achieving  R&M  requirements  in  ASD  program  offices. 

Null  Hypothesis  l.  R&M  management  tools 
associated  with  the  planning  function  have  had  little  or  no 
effect  on  achieving  R&M  requirements  in  ASD  program  offices 

Null  Hypothesis  2.  R&M  management  tools 
associated  with  the  organizing  function  have  had  little  or 
no  effect  on  achieving  R&M  requirements  in  ASD  program 
offices . 
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Null  Hypothesis  3.  R&M  management  tools 
associated  with  the  directing  function  have  had  little  or  no 
effect  on  achieving  R&M  requirements  in  ASD  program  offices. 

Null  Hypothesis  4.  R&M  management  tools 
associated  with  the  controlling  function  have  had  little  or 
no  effect  on  achieving  R&M  requirements  in  ASD  program 
offices . 

Proposition  2.  Program-specific  R&M  requirements  have 
had  little  or  no  influence  on  the  management  of  integrated 
logistics  support. 

Null  Hypothesis  Program-specific  R&M 

requirements  have  had  little  or  no  influence  on  management 
of  maintenance  planning. 

Null  Hypothesis  6.  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  manpower  and  personnel. 

Null  Hypothesis  Program-specific  R&M 

requirements  have  had  little  or  no  influence  on  management 
of  supply  support. 

Null  Hypothesis  8.  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  support  equipment. 

Null  Hypothesis  9.  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  technical  data. 
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requirements  have  had  little  or  no  influence  on  management 
of  computer  resources. 


Null  Hypothesis  11 .  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  training. 

Null  Hypothesis  12 .  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  design  interface. 

Null  Hypothesis  13 .  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  facilities. 

Null  Hypothesis  14 .  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  packaging,  handling,  storage,  and  transportation. 

Null  Hypothesis  15 .  There  is  no  difference  in  the  rank 
order  ASD  senior  level  managers  and  HQ  USAF  leadership  give 
to  the  goals  of  the  R&M  2000  Process. 

Proposition  _3*  An  adequate  level  of  R&M  education  and 
training  is  not  available  within  ASD  program  offices. 

Null  Hypothesis  16 .  An  adequate  supply  of  skilled 
and  experienced  R&M  personnel  does  not  exist  within  ASD 
program  offices. 

Null  Hypothesis  17.  An  immediate  need  for  R&M 
education  and  training  exists  within  ASD  program  offices. 


6 


Null  Hypothesis  18 .  The  R&M  2000  Process  has  had 
little  or  no  effect  on  the  management  of  R&M  in  ASD  program 
offices . 

Figure  1  graphically  depicts  the  objective-proposition- 
null  hypothesis  relationships  described  above. 

Scope 

There  are  five  Air  Force  Systems  Command  product 
divisions  involved  in  the  acquisition  of  new  weapon  systems 
and/or  support  equipment,  avionics,  etc.  They  are: 

1.  Aeronautical  Systems  Division  -  responsible  for 
directing  acquisition  of  aircraft;  and  missiles,  engines, 
simulators,  electronic  warfare  equipment,  and  improved 
avionics  for  these  new  and  existing  air  frames, 

2.  Electronic  Systems  Division  -  responsible  for  the 
acquisition  of  electronic  systems  and  equipment  for  command 
control  communications  and  intelligence  functions  undertaken 
by  the  United  States  Air  Force, 

3.  The  Ballistic  Missile  Office  -  responsible  for  the 
acquisition  of  all  major  end  items  and  sub-systems  for  the 
Air  Force  ballistic  missile  fleet, 

4.  Space  Division  -  responsible  for  the  acquisition  of 
space-based  communications,  navigation,  meteorological,  and 
surveillance  systems;  as  well  as  spacecraft,  launch 
vehicles,  and  all  associated  support  equipment.  And, 

5.  Armament  Division  -  responsible  for  the  acquisition 
of  conventional  air  armament,  as  well  as  the  test  and 
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evaluation  of  armament  and  electronic  combat  systems  and 
associated  equipment  (18:205). 

Not  mentioned  are  the  major  command  units  and 
headquarters  directorates  responsible  for  monitoring  or 
managing  weapon  system  acquisition.  Nor  is  mention  made  of 
the  Air  Force  Operational  Test  and  Evaluation  Center,  the 
special  operating  agency  responsible  for  monitoring  or 
performing  the  operational  testing  of  all  newly  acquired 
systems  and  modernization  of  existing  systems. 

Each  of  these  organizations  has  a  branch  responsible 
for  managing  control  of  reliability,  maintainability,  and 
integrated  logistics  support  concerns.  To  examine  attitudes 
across  the  full  breadth  of  this  universe  would  be  a 
monumental  undertaking  beyond  the  time  constraints  placed  on 
this  researcher. 

Since  this  is  an  exploratory  research  effort  in  support 
of  AFIT's  role  as  the  Air  Force's  CERM  and  since  ASD  is 
located  at  Wr ight-Patterson  along  with  AFIT,  it  seemed  the 
natural  choice  for  keeping  the  scope  of  the  research  at  a 
manageable  level.  Proven  successful,  follow-on  studies 
could  be  performed  using  the  same  research  instrument  within 
these  other  organizations. 
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II.  LITERATURE  REVIEW 


Introduction 

Three  areas  were  emphasized  in  the  literature  review. 
First,  the  historical  development  of  the  R&M  2000 
principles.  Here  it  was  shown  how  senior  DoD  leaders 
arrived  at  what  they  consider  to  be  essential  "building 
blocks"  (25:11)  of  the  R&M  2000  management  process.  Next, 
four  previously  accomplished  research  endeavors  were 
reviewed  which  assessed/critiqued  past  R&M  management 
practices.  One  interesting  finding  was  that  the 
recommendations  made  by  the  researching  agencies  closely 
resemble  the  aforementioned  building  blocks  of  the  R&M  2000 
process.  And  lastly,  a  review  of  accepted  principles  of 
attitude  theory,  as  a  foundation  for  the  direction  of  this 
research  project,  was  undertaken. 

Historical  Development  of  R&M  2000  Principles 

In  January  of  1981,  the  DoD  took  a  new  tack  with 
respect  to  management  of  both  day-to-day  operations  and  the 
business  of  weapon  system  acquisition.  Increased  military 
spending  was  a  reality  as  a  result  of  the  Reagan 
Administration's  push  for  a  build-up  in  the  nation's  defense 
and  the  modernization  of  strategic  forces.  Under  the 
direction  of  Secretary  of  Defense  Casper  Weinberger  and 
Deputy  Secretary  Frank  Carlucci,  a  "plan  of  action"  was 
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devised  to  assist  the  acquisition  community  in  managing  this 

increased  activity.  The  theme  of  this  new  guidance  was 

successful  acquisition  of  a  sufficient  quantity  of  new 

equipment  coupled  with  an  "increase  in  the  quality  of  the 

acquisition  community's  effort... as  well,"  (23:5). 

On  30  April  1981,  Deputy  Secretary  Carlucci  published  a 

memorandum  with  instituted  the  Defense  Acquisition 

Improvement  Program.  The  objectives  of  the  program  were 

to  reduce  costs,  to  make  the  acquisition  process 
more  efficient,  to  increase  the  stability  of 
programs,  and  to  decrease  to  time  it  takes  to 
acquire  military  hardware.  (23:5) 

Secretary  Carlucci  published  32  initiatives  (Actions) 

designed  to  guide  the  DoD  towards  accomplishing  these  goals. 

To  highlight  the  requirement  for  increased  emphasis  on 

quality,  Secretary  Carlucci  points  out  that 

Improved  readiness  is  a  primary  objective  of 
the  acquisition  process,  of  comparable  importance 
to  reduced  unit  cost  or  reduced  acquisition 
time.  Resources  to  achieve  readiness  will  receive 
the  same  emphasis  as  those  required  to  achieve 
schedule  or  performance  objectives. 

-DEPSECDEF  Memorandum,  30  April  1981.  (6:11) 

To  this  end,  five  of  the  32  can  be  directly  related  to 

the  issue  of  reliability  and  maintainability.  They  are: 


1. 

2. 

3. 

4. 

5. 


Action  9--System  Support  and  Readiness, 

Action  12--Funding  for  Test  Hardware, 

Action  16 — Contractor  Incentives  for  Support, 
Action  30--Logist ics  and  Support  Resources,  and 
Action  31--Improved  Reliability  and  Support  (5:7) 
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Actions  9  and  31  require  identifying  readiness 
objectives  for  a  new  weapon  system  very  early  in  the 
acquisition  program,  that  obtaining  these  objectives  is  tied 
directly  to  program  milestones,  and  that  adequate  funding  is 
available  to  ensure  that  reliability  and  maintainability 
is  designed  and  built  into  the  system.  Action  16  directs 
the  program  offices  to  employ  contractual  incentives  for 
obtaining  reliability  and  maintainability  in  design.  This 
requirement  was  re-emphasized  in  an  additional  memorandum 
issued  in  August  1981  <36:6).  This  action  also  gave  rise 
to  an  unwritten  requirement  for  logisticians  to  have  a 
critical  role  in  acquisition  planning  much  sooner  than 
normally  accepted  (36:6).  Action  30  increases  program 
manager  involvement  in  supportability  decisions  on  his 
program.  And  Action  12  ensures  that  adequate  test  articles 
are  made  available  early  in  the  acquisition  process 
( 5 : 9 ) ( 6 : 11 ) .  It  is  generally  understood  that  the  Defense 
Acquisition  Improvement  Program  will  continue  to  be 
emphasized  through  strict  enforcement  of  its  initiatives  for 
some  time  to  come  (5:10). 

Defense  industry  executives  have  taken  a  hard  look  at 
this  new  acquisition  program.  They  have  duly  noted  the  less 
than  enthusiastic  way  acquisition  managers  have  addressed 
supportability  issues  in  the  past  (37:5-6).  They  also  note 
that  it  will  be  in  the  DoD's  best  interest  to  spend  the 
additional  dollars  required  to  improve  supportability  at  the 
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•front-end"  of  the  acquisition  in  order  to  reap  the  rewards 
of  decreased  total  life  cycle  costs  in  the  long  run 
(37:6-7) . 

Previous  Research 

Prior  to  the  institution  of  the  Defense  Acquisition 
Improvement  Program,  numerous  studies  were  undertaken  to 
examine  operations  within  the  weapon  system  acquisition 
arena.  Here  are  but  four  of  those  studies  and  there 
results/recommendations . 

The  AFALD  Study .  In  September  1978,  Lt  Gen  John 
G.  Albert,  then  Commander  of  the  Air  Force  Acquisition 
Logistics  Division  (AFALD.,  now  the  Air  Force  Acquisition 
Logistics  Center — AFALC),  directed  a  survey  be  administered 
to  all  Deputy  Program  Managers  for  Logistics  ( DPMLs )  as  a 
means  of  ascertaining  to  what  extent  contractual  reliability 
requirements  were  being  adhered  to  in  ongoing  Air  Force 
acquisition  programs  (32:1).  It  is  the  responsibility  of 
the  DPML  to  ensure  issues  pertaining  to  reliability, 
maintainability,  and  integrated  logistics  support  on  his 
program  are  not  overlooked  or  shortchanged  throughout  the 
acquisition  process.  The  results  of  that  study  highlighted 
deficiencies  in  three  areas: 

1.  A  general  lack  of  capability  to  accurately  identify 
in  which  stage  of  the  acquisition  process  a  program  actually 
was.  As  a  result,  requirements  for  system  reliability  which 
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were  to  be  emphasized  at  particular  stages  were  often 
overlooked, 

2.  A  general  inability  to  differentiate  between  the 
numerical  measures  of  reliability  and  maintainability, 
leading  to  potential  ineffective  management  of  the 
reliability  issues  pursuant  to  the  program,  and 

3.  A  lack  of  knowledge  as  to  which  reliability  tasks 
are  appropriate  for  emphasis  in  a  given  stage  of  the 
acquisition  process  (32:4-25). 

It  was  also  pointed  out- that  continuing  emphasis  on 
education  in  the  area  of  R&M  was  required.  Several 
documents,  sources  of  classroom  education,  and  the  need  to 
tap  the  expertise  of  experienced  acquisition  personnel  were 
cited  as  means  of  remedying  the  situation  (32:25). 

The  DSB  Study.  In  1981,  a  study  of  the 
operational  readiness  of  high  performance  systems  was 
published  by  the  Defense  Science  Board  (DSB).  The  study 
recommended  a  need  to  design  reliability  into  systems  from 
the  start,  and  to  mature  that  capability  prior  to  full-rate 
production  (33:Sec  1,1).  It  was  also  noted  that  as  acquired 
systems  became  more  and  more  complex,  DOD  would  have  to 
demand  the  acquisition  process  put  increased  management 
emphasis  on  R&M  issues  (33:Sec  I,  1).  Although  the  Carlucci 
Initiatives  (36:7)  took  the  first  steps  in  this  direction, 
the  demonstrated  progress  of  the  acquisition  community  in 
keeping  R&M  emphasis  high  indicated  less  than  full  support 
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of  the  program  (33:Sec  I,  1-2).  In  their  defense,  however, 
it  was  noted  that  accelerating  acquisition  programs  to 
respond  to  the  (Reagan)  Administration's  objective  of 
fielding  hardware  faster  was  perturbating  this  already  bad 
situation  (33:  Sec  I,  2). 

The  OSD/IDA  Study .  In  November  1983,  the 
Institute  for  Defense  Analyses  (IDA)  undertook  an  R&M  study 
for  the  Office  of  the  Secretary  of  Defense  (OSD)  in  response 
to  the  DSB  findings.  As  a  result,  ten  areas  of  improvement 
were  recommended.  Among  the  most  critical  of  these  were  the 
following  seven: 

1.  The  R&M  Demonstration  Program — wherein  the 
contractor  is  required  to  physically  demonstrate  that  time 
and  procedures  required  for  system  maintenance  are  as 
accurate  as  outlined  in  system  specifications,  and  that 
personnel  of  the  designated  maintenance  skill  level(s)  are 
capable  of  performing  the  task, 

2.  R&M  Standards--An  up  front,  predetermined  level  of 
R&M  performance  is  established  during  the  weapon  system 
concept  validation  stage,  supported  by  inputs  from  the  user 
and  acquisition  community  logisticians. 

3.  Management  Incentives--A  means  of  rewarding 
program  office  managers  for  focusing  on  issues  pertaining  to 
R&M  in  their  interactions  with  contractors. 

4.  New  System  Maturat ion--wherein  a  plan  is 
established  to  track  the  performance  of  those  systems  whose 
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reliability  is  heavily  dependent  upon  "new  and  evolving 
technology"  after  it  is  fielded  for  correcting  deficiencies 
resulting  from  previously  unforeseen  operational  demands, 

5.  Collection  and  Use  of  Field  R&M  Data — by  system 
development  engineers.  In  the  past,  little  attention  was 
given  to  the  suggestions  and  recommendations  of  users  as  to 
ways  in  which  either  initial  system  design  or  design  changes 
can  facilitate  weapon  system  field  support, 

6.  R&M  Training  for  Managers — Making  ample  and 
productive  use  of  the  myriad  of  opportunities  for  providing 
acquisition  community  personnel  with  professional  education 
and  training  in  R&M  management. 

7.  Diagnostics — Placing  greater  emphasis  on  reducing 
the  deficiencies  in  reporting  accuracy  and  "false  alarms" 
with  respect  to  built-in  test  equipment  and  subcomponents 

( 33 : Sec  II). 

The  ASP  Survey .  In  November  1987,  ASD  completed 
an  industry  survey  which  yielded  information  as  to  policies, 
practices,  and  procedures  the  defense  contractor  community 
was  using  to  increase  the  emphasis  put  on  R&M  -  the  major 
focus  being  placed  on  engineering  areas.  They  then  compared 
these  findings  to  their  in-house  policies,  practices,  etc., 
and  identified  several  disconnects.  Areas  where  improvement 
was  recommended  were: 

1.  More  internal  training  on  R&M  principles  and 


applications, 


2.  Increased  exposure  of  contractors  to  the  use  of 
weapon  systems  in  the  operational  environments, 

3.  Integrating  logistics  support  requirements  in  the 
system  design  process, 

4.  Increased  quality  control  for  replacement  parts, 

and 

5.  Pursuing  aggressive  implementation  of  integrity 
programs,  the  goal  of  which  is  to  design  a  structure  that 
does  not  break  down  —  as  opposed  to  the  traditional  Mean 
Time  Between  Failure  goal.  Critical  to  this  approach  is  the 
integration  of  R&M  requirements  into  the  integrity  program 
approach  (22). 

In  short,  the  theme  common  to  all  these  recommendations 
is  the  need  for  increased  user  participation  and  earlier 
emphasis  on  R&M  requirements  in  system  design  and  support, 
and  improved  education  and  training. 

The  R&M  2000  Process 

Following  is  an  overview  of  R&M  2000,  the  Air  Force's 
initiative  to  ensure  its  acquisition  managers  are  assisted 
as  much  as  possible  in  the  management  of  weapon  acquisition 
in  accordance  with  the  Carlucci  Initiatives  of  the  Defense 
Acquisition  Improvement  Program.  Air  Force  R&M  2000  goals, 
in  order  of  importance,  are: 

1.  Increase  Combat  Capability — by  using  R&M  as  a  means 
of  improving  performance  over  time. 


2.  Increase  Survivability  of  the  Combat  Support 
Structure — by  as  much  as  possible  "breaking  the  logistics 
tether, " 

3.  Decrease  Mobility  Requirements  Per  Unit — a  combat 
support  structure  that  is  not  needed  does  not  have  to  be 
moved  or  mobilized, 

4.  Decrease  Manpower  Requirements  Per  Unit  of  Output — 
the  fewer  the  number  of  personnel  required  to  support  a 
weapon  system,  the  more  personnel  there  are  available  for 
duties  elsewhere  in  the  support  structure.  This  gives  rise 
to  the  concept  of  viewing  reliability  and  maintainability  as 
a  "force  multiplier,"  and 

5.  Decrease  Costs — through  well  thought  out  design  and 
endeavoring  to  do  things  right  the  first  time  (25:2-7). 

To  assist  Air  Force  managers  in  attaining  these  goals, 
five  management  principles,  and  21  "building  blocks"  to 
support  those  principles  were  established.  They  are: 

1.  Involvement--highlighting  that  change  cannot  take 
place  without  management's  commitment  to  it, 

2.  Motivation--of  industry  to  capitalize  on  industrial 
capability  to  increase  combat  capability.  The  three 
building  blocks  associated  with  this  principle  are: 

a.  Source  Selection--R&M  must  be  singled  out  as  a 
specific  evaluation  criteria. 
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b.  Performance-based  Progress — low  performance 
means  no  money! 

c.  Incentives  (during  the  design  phase)  and 
Warranties  (during  field  performance). 

3.  Requirements — for  operational  performance  must  be 
communicated  clearly.  Associated  building  blocks  are: 

a.  Clear  Requirements — ensure  they  are  mission- 
based  and  operationally  oriented. 

b.  Technician  Transparency--attempting  to  reduce 
the  number  of  specialists  required  to  maintain  increasingly 
capable  systems. 

c.  Simplification — designing  a  system/subsystem  so 
that  it  will  be  easier  and  cheaper  to  produce. 

d.  Modularity — packaging  components  and  software 
into  self-contained  units. 

e.  R&M  Plans — should  reflect  an  organization's 
goals  and  objectives,  and  the  strategy  for  attaining  them. 

f.  Company  Policy  and  Practices — documenting  the 
contractor's  commitment  to  R&M. 

4.  Design  and  Growth — consider  R&M  needs  from  the 
start,  and  allow  them  to  mature  with  the  system.  In  order 
to  support  this  principle,  provided  are: 

a.  Systems  Engineering  Process — committing  the 
acquisition  team  to  delivering  products  that  meet  user  needs 
at  a  reasonable  cost. 
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b.  Allocation  and  Prediction — establishing  a 
hierarchy  of  requirements  for  designers,  then  making  and 
tracking  estimates  as  to  whether  or  not  requirements  will  be 
achieved . 

c.  Analysis — must  be  a  key  activity  in  design  and 
development . 

d.  Growth-  Management — a  "stair  step"  approach  to 
managing/tracking  the  growth  of  R&M  capability  in  a  weapon. 

e.  Parts  Selection — use  only  the  best  available. 

f.  Derating — restricting  stresses  on  components  to 
levels  below  their  rated  operational  limits. 

g.  Computer-Aided  Tools — make  use  of  them  in  order 
to  increase  efficiency  in  the  design  and  production 
processes. 

h.  Test,  Analyze,  and  Fix — must  be  an  ongoing 
methodology  whereby  deficiencies  are  analyzed,  corrected, 
and  retested  as  they  occur  to  ensure  failure  causes  are 
completely  removed.  And, 

5.  Preservation — ensuring  inherent  R&M  capabilities 
are  preserved  during  production  and  maintained  in  the 
operational  environment.  Supporting  this  principle  are: 

a.  Variability  Reduction  Program--with  a  goal  of 
developing  a  highly  capable  manufacturing  process  which 
produces  uniform,  defect-free  items  which  meet  design 
requirements . 
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b.  Environmental  Stress  Screening — simulating 
assembly  operating  environment  conditions  in  order  to 
precipitate  failure  in  the  factory  and  eliminate  failure  in 
the  field. 


c.  System  Testing — the  continuous  assessment  of 
performance  in  actual  or  simulated  conditions. 

d.  Feedback--f acilitating  the  flow  of  information 
from  the  field  to  design  engineers  for  assessing  R&M 
performance  and  for  building  a  database  for  future  product 
and/or  process  improvement  (25:11-103). 


Figure  2,  taken  from  the  R&M  2000  Process  pamphlet, 
shows  the  overall  relationships  between  the  five  management 
principles  and  21  building  blocks.  Top  management 
involvement  is  the  "blanket"  which  envelopes  all  other 


principles  and  building  blocks,  emphasizing  its  importance 
to  the  successful  implementation  of  the  others. 


MANAGEMENT  INVOLVEMENT 

MOTIVATION 

•  SOURCE  SELECTION 

•  PERFORMANCE-EASED  PROGRESS 

•  SCeiTIVES  ANO  WARRANTIES 

REQUIREMENTS 

DESIGN  AND  GROWTH 

PRESERVATION 

•  CLEAR  REQUIREMENTS 

•  TECHNICIAN 
TRANSPARENCY 

•  3MPUPICATTON 

•  MODULARITY 

•  RAM  PLANS 

•  COMPANY  POUCY 
ANO  PRACTICES 


•  SYSTEMS  ENONEERWQ 
PROCESS 

•  ALLOCATION  a  PREDICTION 

•  ANALYSIS 

•  GROWTH  MANAGEMENT 

•  PARTS  SELECTION 

•  OERATWQ 

•  COMPUTER-AGED  TOOLS 

•  TEST.  ANALYZE.  ANO  FIX 


•  RAM  2000  VARIA8AJTY 
REDUCTION  PROGRAM 

•  RAM  2000  ENVIRONMENTAL. 
STRESS  SCREEN**} 

•  SYSTEM  TESTWO 

•  FEEDBACK 


Figure  2  -  The  R&M  2000  Principles  and  Building  Blocks 
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Attitude  Theory 


Since  the  purpose  of  this  study  is  to  obtain 

information  on  the  attitudes  and  opinions  of  senior  level 

ASD  managers,  a  working  familiarity  with  accepted  theory 

detailing  what  constitutes  attitudes  and  how  they  are 

developed  is  a  prerequisite  to  construction  of  a  useful 

research  tool.  According  to  Doobs,  attitude  is  defined  as 

...  an  implicit  response  with  drive  strength 
which  occurs  with  the  individual  as  a  reaction 
to  stimulus  patterns  and  which  affects  subsequent 
overt  responses.  (11:43) 

He  further  cautions  on  using  attitudes  alone  as  a 
predictor  of  behavior  (1:9) (11:44) . 

Peak  defines  attitude  as  a  product  of  our  perceptions 
displaying  affective  properties  as  manifest  externally  by 
our  likes  and  dislikes,  or  by  what  we  favor  or  not  favor 
(26:67) . 

Fishbein  and  Ajzen  agree  in  context  with  the  above 
definitions  and  go  a  step  further  to  subdivide  attitude  into 
three  components:  l)a  cognitive  component  dealing  with  the 
individual's  beliefs  and  ideas;  2)an  affective  component, 
referring  to  the  individual's  feelings  towards  the  issue  at 
hand,  and  3)a  behavioral  intention  component,  which  refers 
to  how  the  individual  will  tend  to  behave  towards  the 
person,  object,  etc.,  as  a  function  of  their  affective 
component  (9:75).  Actual  behavior  is  the  end  result  of  the 
translation  of  behavioral  intention  into  action. 
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Crespi  takes  a  similar  view  defining  attitude  as 
"predispositions  to  [consistently]  behave  in  specific  ways 
to  specific  stimuli"  (8:10).  He  goes  on  to  state  that 
because  of  the  non-observability  of  predispositions,  a  means 
of  inferring  attitude  must  be  developed.  He,  therefore, 
builds  on  his  previous  definition  by  extrapolating  that  an 
attitude  can  be  considered  "an  inferred  underlying  structure 
of  cognition,  frame  of  reference,  evaluation,  and  affect" 
(8:10).  By  cognition,  Crespi  means  an  "awareness  and 
knowledge"  of  the  subject  (8:5).  Frame  of  reference  refers 
to  the  level  of  significance  the  subject  has  in  the  mind  of 
the  individual  (8:6).  Evaluation  accounts  for  the  positive 
or  negative  reaction  an  individual  has  when  confronted  by 
the  subject  (8:7).  And  finally,  affect  can  be  considered  as 
the  measure  of  magnitude  of  that  positive  or  negative 
reaction  (8:5). 

Based  upon  the  author's  consolidation  of  the  above 
mentioned  experts'  definitions  of  attitude,  the  following 
operational  definition  of  attitude  was  developed  for  use  in 
this  research: 

The  inferred  predisposition  of  an  individual 
towards  a  given  subject  as  overtly  manifest 
by  response  to  various  questions  on  the  subject; 
assumed  to  be  the  product  of  the  individual's 
beliefs,  ideas,  values,  and  experience. 

The  challenge  was  to  design  the  research  instrument  to 
gather  as  much  information  as  possible  on  respondents' 
"underlying  structures  of  cognition,  frame  of  reference, 


evaluation,  and  affect"  in  hopes  of  building  a  legitimate 
word  picture  of  attitudes  towards  the  R&M  2000  process  and 
its  management. 
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Ill .  METHODOLOGY 


Introduction 

Ex  post  facto  research  was  conducted  through  use  of 
a  mail  questionnaire  (survey).  In  granting  their  approval 
to  perform  this  research,  ASD  requested  that  as  little  time 
as  possible  be  taken  from  the  daily  schedules  of  all 
respondents.  As  a  result,  the  survey  method  was  selected. 

Table  I  shows  a  breakdown  of  survey  questions  and  the 
propositions  and  null  hypotheses  they  serve  to  answer,  as 
well  as  the  associated  level  of  data  each  question  provides. 
The  survey  instrument  is  presented  in  Appendix  A.  Formal 
approval  for  its  use  was  granted  by  the  Air  Force  Military 
Personnel  Center  ( AFMPC ) .  ASD  Headquarters  subsequently 
granted  approval  for  its  distribution. 

The  Population 

For  the  purpose  of  this  study,  the  population  was 
defined  to  address  senior  level  ASD  managers,  including 
military  and  civilian  managers  assigned  to  the  168  ongoing 
ASD  acquisition  programs  who  hold  the  positions  of  program 
manager/director,  deputy  program  manager  for  logistics, 
director/deputy  for  engineering,  deputy  for  test  and 
evaluation,  or  integrated  logistics  support  manager.  At 
present,  this  population  numbers  approximately  600. 
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TABLE  I 


SURVEY  QUESTION  ASSIGNMENTS  TO  PROPOSITIONS  AND  HYPOTHESES 


Prop. 

# 

Hypo. 

* 

Ques 

# 

• 

Area  Addressed 

Level  of 
Data 

1 

I 

la  - 

ig 

The  Planning  function 

Interval 

2 

lh  & 

lm 

The  Organizing  function 

Interval 

3 

li,ll. 

In  &  lo 

The  Directing  function 

Interval 

4 

lj» 

lp  - 

lk, 

lu 

The  Controlling  function 

Interval 

2 

5 

2a 

Maintenance  Planning 

Interval 

6 

2b 

Manpower  &  Personnel 

Interval 

7 

2c 

Supply  Support 

Interval 

8 

2d 

Support  Equipment 

Interval 

9 

2e 

Technical  Data 

Interval 

10 

2f 

Computer  Resources 

Interval 

11 

2g 

Training 

Interval 

12 

2h 

Design  Interface 

Interval 

13 

2i 

Facilities 

Interval 

14 

2j 

Packaging,  Handling 
Storage,  &  Transport¬ 
ation 

Interval 

15 

3 

The  ranking  of  R&M 

2000  objectives 

Ordinal 

3 

16 

6 

R&M  skill/experience 
level 

Ordinal 

17 

7 

Need  for  R&M  education 
&  training 

Ordinal 

18 

8 

Effectiveness  of  the 

R&M  2000  process 

Interval 
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The  Sample 


The  sample  contained  members  of  the  population  assigned 
to  the  specific  duties  mentioned  above.  ASD  approva1  for 
distribution  of  the  survey  was  contingent  upon  limiting  the 
sample  to  include  only  the  16  major  system  program  offices. 
What  this  resulted  in,  therefore,  was  a  non-probability 
judgement  sample  of  62  possible  respondents.  The  use  of 
judgement  sampling  was  considered  appropriate  in  this 
situation  (12:280)  due  to  the  exploratory  nature  of  the 
research.  Additionally,  non-probability  sampling  is 
considered  appropriate  if  the  entire  population  is  not 
available  for  study  (12:279).  Given  the  limitations  placed 
on  this  study  by  ASD,  this  was  also  a  valid  point. 

Since  the  names  of  all  sample  members  were  known, 
surveys  were  distributed  through  ASD's  Office  of  the  Chief 
of  Staff.  Completed  questionnaires  were  return  mailed  to 
the  author. 


Validity 


Internal .  During  survey  development,  several  experts 
in  the  area  of  R&M  were  given. the  opportunity  to  critique 
and  recommend  changes  to  ensure  the  instrument  measured  what 
it  was  intended  to  measure.  This  expert  opinion  was  gleaned 
from  the  faculty  of  the  Air  Force  Institute  of  Technology 
School  of  Systems  and  Logistics  (AFIT/LS),  the  Air  Force 
Acquisition  Logistics  Center,  and  the  ASD  Headquarters 
staff.  Through  this  process,  adequate  coverage  of  the 
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problem  by  the  survey  instrument  was  ensured  (12:95).  Also 
validated  in  this  fashion  was  the  survey  design  itself 
(12:95)  via  AFIT/LS  faculty  assigned  to  the  Behavioral 
Sciences  department.  AFMPC  was  the  final  validating 
authority. 

External .  Prevention  of  respondent  exposure  to  the 
survey  prior  to  its  distribution,  selection  of  respondents 
from  the  same  AFSC  product  division,  and  the  guarantee  of 
anonymity  enabled  the  author  to  "generalize  findings  to  and 
across  [population]  times,  settings,  and  persons"  (12:118). 

Statistical  Tests 

The  hypothesis  testing  was  performed  using  SAS  (34)  and 
Lotus  1-2-3  (3).  For  testing  purposes,  the  traditional 
null/alternate  hypothesis  relationship  was  used  (24). 

Comparison  of  Means,  Medians ,  and  Modes .  The  Lickert 
scale  provided  interval  data  capable  of  using  the  sample 
mean  as  both  a  measure  of  central  tendency  and  test 
statistic.  Table  II  shows  the  breakdown  of  numeric  and 
contextual  values  for  the  survey  Lickert  scale.  However,  in 
order  to  avoid  inadvertently  overlooking  abnormal  data 
dispersions  (such  as  a  bimodal  distributions)  prior  to 
performing  the  mathematics  of  calculating  a  mean,  the  median 
and  mode  were  first  ascertained  (24:57-87).  If  reasonably 
close,  or  if  bimodality  did  not  exist,  then  the  mean  was 
used.  If  not,  then  an  attempt  was  made  to  find  why  the 
abnormal  dispersion  existed. 
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TABLE  II 


MEANING  OF  LICKERT  SCALE  VALUES 


Numeric 

Contextual 

Value 

Meaning 

0.00  -  1.75 

little  or  no  influence 

(or  effect) 

1.76  -  3.50 

minimal  influence  (or 

effect ) 

3.51  -  5.25 

moderate  influence  (or 

effect ) 

5.26  -  7.00 

significant  influence 

(or  effect) 

This  procedure  was  applied  to  questions  relating  to 
null  hypotheses  1  through  14,  and  18.  Given  a  reasonably 
normal  distribution,  total  sample  means  were  calculated 
using  the  formula: 


(  24:60  ) 


where  x;  is  the  ith  measurement  and  n  is  the  number  of 
sampling  units  from  stratum  i.  Total  sample  standard 
deviations  were  calculated  using  the  formula: 


S  = 


(  24:76) 


The  means  and  standard  deviations  were  used  in  the 
hypothesis  testing.  The  associated  test  statistic  was  a 
z-value  calculated  using  the  formula: 

X  - 


z 


(24:293) 


f 


t  ' 


This  z-value  was  compared  to  a  za  -value  obtained 
from  a  normal  distribution  table  of  z-values.  The  z0-value 
was  a  constant  1.28,  which  reflects  an  alpha  ( & )  of  0.10. 
The  calculated  z-value  must  be  greater  than  the  za-value  in 
order  for  the  null  hypothesis  to  be  rejected  (24:293). 

Table  III  lists  the  null/alternate  hypothesis  pairs  to  be 
tested  for  acceptance/rejection  in  support  of  null 
hypotheses  1  through  14,  and  null  hypothesis  18. 

Thurstone 1 s  Case  V  Scaling .  Used  for  Hypothesis  15, 
this  technique  permits  construction  of  a  unidimensional 
interval  scale  using  responses  obtained  from  such  tests  as 
paired  comparisons.  Two  assumptions  accompany  use  of  this 
model.  They  are: 

1.  The  variance  between  items  paired  for  comparison  is 
the  same  for  all  item  pairs,  and 

2.  The  manner  in  which  the  respondent  makes  the 
selection  of  one  of  the  items  in  a  pair  is  the  same  each 
time  a  selection  is  made  (8:180-182). 

An  R*  value  was  calculated  as  a  function  of  a  Thurstone 
Z  value  obtained  from  a  supplied  chart.  The  Z  values  were 
determined  as  a  result  of  the  proportionate  number  of  times 
a  given  item  was  preferred  over  all  other  alternatives. 

Based  on  R*  values  calculated  for  each  of  the  comparisons, 
an  interval  rank  ordering  was  accomplished  (8:182-185). 
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From  analysis  of  this  rank  ordering  it  was  determined  if 
null  hypothesis  15  was  accepted  or  rejected. 


TABLE  III 


Null/Aiternate  Hypotheses  used  For  Tests 


Null  Null 

Hyp  Hypothesis  Statement 

One-Tailed 

Test 

Test 

Statistic 

1 

R&M  management  tools  associated  with 
the  planning  function  have  had  little 
or  no  effect  on  achieving  R&M 

Ho:  /A?  1.75 

Given  that 
z„  *  1.28; 

requirements  in  ASD  program  offices. 

Ha:  ft>  1.75 

Z  >  Za 

2 

R&M  management  tools  associated  with  the 
organizing  function  have  had  little 
or  no  effect  on  achieving  R&M 

Ho:  ft$  1.75 

Given  that 
z  a  =>  1.28: 

requirements  in  ASD  program  offices. 

Ha:  ft  >  1.75 

z  >  Za 

3 

R&M  management  tools  associated  with 
the  directing  function  have  had  little 
or  no  effect  on  achieving  R&M 

Ho:  /A  ■  1.75 

Given  that 
Zo  *  1.28; 

requirements  in  ASD  program  offices. 

Ha:  ft  >  1.75 

Z  >  Za 

4 

R&M  management  tools  associated  with  the 
controlling  function  have  had  little 
or  no  effect  on  achieving  R&M 

Ho:  /A?  1.75 

Given  that 

Za  »  1.28; 

requirements  in  ASD  program  offices. 

Ha:  ft  >  1.75 

z  >  z„ 

5 

Program-specific  R&M  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho:  ft*  1.75 

Za  ■  1.28; 

management  of  maintenance  planning. 

Ha:  /A  >  1.75 

Z  >  Za 

6 

Program-specific  R&M  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho:  ft*  1.75 

Za  ■  1.28; 

management  of  manpower  and  personnel. 

Ha:  ft  >  1.75 

Z  >  Za 

7 

Progran-soecific  R&M  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho:  ft*  1.75 

Za  *  1.28; 

management  of  supply  support. 

Ha:  ft  >  1.75 

Z  >  Zg 

8 

Program-specific  R&M  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho:  ftt  1.75 

zg  ■  1.28; 

management  of  support  equipment. 

Ha:  ft  >  1.75 

Z  >  Zo 

9 

Program-specific  R&M  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho:  ft  s  1.75 

z0  -  1.28; 

management  of  technical  data. 

Ha:  ft  >  1.75 

Z  >  Za 

10 

Program-specific  R&M  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho:  P*  1.75 

z a  -  1.28: 

management  of  computer  resources. 

Ha:  ft  >  1.75 

Z  >  Za 

TABLE  III  -  Continued 


Null/Alternate  Hypotheses  Used  Foe  Tests 


Null  Null 

One-Tailed 

Test 

Hyp  Hypothesis  Statement 

Test 

Statistic 

11  Program-specific  RSM  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho: 

/A? 

1.75 

Zq  m  1.28; 

management  of  training. 

Ha: 

1.75 

Z  >  Zq 

12  Program-specific  RSM  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho: 

1.75 

z,  -  1.28; 

management  of  design  interface. 

Ha: 

1.75 

Z  >  Z0 

13  Program-specific  R&M  requirements  have 

Given  that 

had  little  or  no  influence  on 

Ho: 

1.75 

Zq  *  1.28; 

management  of  facilities. 

Ha: 

1.75 

Z  >  Zq 

14  Program-specific  R&M  requirements  have 

had  little  or  no  influence  on 

Given  that 

management  of  packaging,  handling. 

Ho: 

1.75 

Za  »  1.28; 

storage  and  transportation. 

Ha: 

1.75 

Z  >  Zq 

15  There  is  no  difference  in  the  rank  order 

ASD  senior  level  managers  and  HQ  OSAF 

See 

leadership  give  to  the  goals  of  the 

Criteria 

R&M  2000  Process. 

N/A 

Test 

16  An  adequate  supply  of  skilled  and 
experienced  R&M  personnel  does  not 
exist  within  asd  program  offices. 


Ho:  Pi  »  p2  *  0.50 
Ha:  A  Preference 
Exists 


17  An  immediate  need  for  R&M  education  Ho:  p,  ■  p2  ■  p3 
and  training  exists  within  ASD  program  p4  •  0.25 

offices.  Ha:  A  preference 

Exists 


18  The  R&M  2000  Process  has  had  little  or 

no  effect  on  the  management  of  R&M  in  Ho:. ft*  1.75 

ASD  program  offices.  Ha:  fl>  1.75 


Given  that 
x§-  2.71; 
*2>*2 


Given  that 
*5-  ,6.25; 
x1  >xj 


Given  that 
Z«  «  1.28; 
2  >  Zo 


Chi-Squared .  To  be  used  for  null  hypotheses  16  and  17. 
Ordinal  count  data  of  a  multinominal  distribution  was 
collected  in  questions  6  and  7  in  support  of  testing  of 
these  hypotheses.  The  associated  properties  of  this 
distribution  are: 

1.  The  experiment  consists  of  n  identical  trials, 

2.  There  are  k  possible  outcomes  to  each  trial, 

3.  The  probabilities  of  the  k  outcomes  (p1,p2,.  .  .pK) 

remain  the  same  from  trial  to  trial, 

4.  The  trials  are  independent, 

5.  The  random  variables  of  interest  are  the  counts, 

n1fn2,.  .  .n„  in  each  of  the  k  cells  (24:790), 
and 

6.  The  sample  size  n  will  be  large  enough  so  that,  for 

every  cell,  the  expected  count,  E(n5  ),  will  be 
equal  to  or  greater  than  five  (24:792). 


The  chi-squared  test  statistic  based  on  the  expected 
outcomes  of  type  i  is  stated  as 


[  n ;  -  E  ( n;  )  ] 2 
E(n;  ) 


(24:792) 


Using  k-1  degrees  of  freedom  and  a  predetermined  alpha, 

2  X2 


the  chart  Xa  value  must  be  less  than  the  calculated 


value  in  order  to  reject  the  null  hypothesis.  Two  different 
Xavalues  were  required.  The  X2 


value  calculated  for  null 


hypothesis  16  was  tested  against  a  Xa  of  2.11,  based  upon  an 
Gt  of  0.10  and  1  degree  of  freedom.  The  X2  value  calculated 
for  null  hypothesis  17  was  tested  against  a  Xa  of  6.25, 
based  on  an  (t  of  0.10  and  3  degrees  of  freedom.  Given 
rejection  of  the  null,  criterion  test  considerations  for  the 
two  null  hypotheses  must  be  satisfied  before  the  Chapter  I 
null  hypotheses  can  be  rejected.  The  null/alternate 
hypothesis  pairs  to  be  tested  for  acceptance/rejection  in 
support  of  null  hypotheses  16  and  17  can  also  be  found  in 
Table  III. 

Nominal  Data  and  Solicited  Answers .  Questions  4  and  5 
supplied  nominal  level  data.  With  respect  to  this  data  as 
useful  to  this  research,  what  was  required  was  a  measure  of 
frequency  of  use  of  each  of  the  listed  categories.  This 
data  is  best  displayed  in  histogram  form  (24:34-39).  In  the 
case  where  examples  are  solicited,  those  examples  served 
to  further  clarify  the  reason  behind  a  given  response,  as 
well  as  provide  a  database  from  which  additional  credence 
was  given  to  any  drawn  inferences. 

Non-Response,  Not  Applicable ,  and  "I  Don 1 1  Know" 

Answers .  As  with  any  research  conducted  by  mail  survey, 
several  answers  were  left  unmarked,  some  marked  "N/A"  or 
not  applicable,  and  some  responded  to  with  an,  "I  don't 
know,"  or,  "I'm  unfamiliar  with  this,"  response.  Proper 
handling  of  these  situations  was  necessary  to  maintain 
integrity  in  research  results/analysis.  Consequently,  in 
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the  calculation  of  sample  means,  the  n  value  in  the  formula 
denominator  reflected  only  the  number  of  responses  received 
to  that  question — not  the  total  number  of  survey  respondents 
(which  may  be  less  if  some  respondents  did  not  answer  the 
question).  All  "N/A"  responses  to  Lickert-type  questions 
were  given  a  value  of  one.  It  was  the  researcher's  opinion 
that  "N/A"  is  an  equivalent  response  to  "Not  Effective"  or 
"No  Influence"  because  if  a  concept/technique  does  not  apply 
to  a  given  organization's  operation,  it  had  no  influence  or 
effect  upon  that  organization.  In  order  to  handle  the  "I 
don't  know/am  unfamiliar"  responses,  a  new  scale  value  of 
zero  was  added.  All  respondents  who  answered  a  question 
with  either  of  the  aforementioned  responses  were  scored  a 
zero  for  that  response.  The  zero  was  also  included  in  the 
calculation  of  the  sample  mean.  This  tactic  was  used 
because,  based  upon  discussions  during  survey  validation,  it 
was  determined  by  the  researcher  that  the  population 
targeted  for  this  research  should  be  familiar  with  all  the 
concepts/techniques  outlined  for  response  in  the  survey 
instrument.  Therefore,  an  "I  don't  know/am  unfamiliar" 
response  is  a  significant  piece  of  data  which  should  not  be 
overlooked.  A  four.th  anomaly  was  the  "I  do  not  understand 
the  question,"  response.  When  this  occurred,  the  answer  was 
treated  as  a  non-response.  The  researcher's  rationale  for 
this  was  that  with  the  design  of  any  survey  instrument, 
there  is  always  a  possibility  of  vagueness  or  legitimate 


lack  of  understanding  of  a  question  by  a  small  percentage  of 
respondents.  It  is  the  researcher's  opinion  that  in  this 
situation  it  would  be  an  erroneous  assumption  that  the 
respondent  was  unfamiliar  with  the  concepts/techniques 
presented. 

Criteria  Tests 

For  survey  questions  one,  two,  eight,  and  nine,  data 
was  collected  via  7-point  Lickert  scale.  Question  three 
utilized  paired  comparisons.  Question  six  called  for  a  yes 
or  no  response.  Question  seven  was  multiple  choice;  and 
questions  four  and  five  required  selection(s)  from  a  list  of 
possible  choices. 

To  reject  null  hypotheses  one  through  four,  the 
mean  obtained  by  grouping  the  responses  for  all  survey 
questions  associated  with  that  hypothesis  must  be 
statistically  greater  than  1.75,  as  proven  by  the  resultant 
z-value  being  greater  than  1.28.  For  Proposition  1  to  be 
rejected,  three  of  the  four  associated  null  hypotheses  must 
be  rejected.  To  reject  null  hypotheses  five  through 
fourteen,  the  mean  response  associated  with  each  question 
must  also  be  statistically  greater  than  1.75,  as  proven  by 
the  resultant  z-value  being  greater  than  1.28.  For 
Proposition  2  to  be  rejected,  six  of  the  ten  associated 
null  hypotheses  must  be  rejected.  For  null  hypothesis  15  to 
be  rejected,  no  difference  must  exist  in  the  way  HQ  USAF  and 
ASD  senior  managers  rank  the  goals  of  R&M  2000.  For 


r,  .1 

.  -  t 


m 


1 


purposes  of  this  research,  a  difference  will  exist  if 
"increasing  combat  capability"  was  not  ranked  first  and 
"decreasing  costs"  was  not  ranked  last.  The  remaining  three 
can  differ.  For  null  hypothesis  16  to  be  rejected,  the 
proportion  of  all  respondents  who  answered  the  associated 
question  "yes"  must  be  greater  than  0.50.  For  null 
hypothesis  17  to  be  rejected,  the  proportion  of  respondents 
who  answered  "immediate"  to  question  7  must  be  less  than  the 
proportions  of  the  other  three  possible  responses.  For 
Proposition  3  to  be  rejected,  both  null  hypotheses  16  and  17 
must  be  rejected.  And  finally,  for  null  hypothesis  18  to  be 
rejected,  the  mean  response  to  the  associated  question  must 
be  statistically  greater  than  1.75,  as  proven  by  the 
resultant  z-value  being  greater  than  1.28. 


P- 
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IV.  DATA  ANALYSIS 


Introduction 

This  chapter  summarizes  results  from  the  data  analysis 
performed  on  all  responses  to  the  administered  mail  survey. 
The  final  survey  response  rate  was  74%  (or  46  of  62). 
Figures  3  through  9  show  the  demographic  breakdown  of  the 
respondents.  All  calculations  and  figures,  unless 
otherwise  indicated,  are  based  upon  total  respondent  group 
statistical  analyses. 

Presentation  Format 

Analyzed  data  will  be  organized  to  follow  the 
proposition-null  hypothesis  order  outlined  in  Chapter  I, 
Figure  1.  The  presentation  format  to  be  used  for  all 
Lickert  scale  and  multiple  choice  questions  begins  with  a 
statement  of  the  proposition  in  question.  This  is  followed 
by  the  analysis  of  each  hypothesis  supporting  that 
particular  proposition.  The  format  for  the  hypotheses 
analyses  is  as  follows: 

1.  Statement  of  the  hypothesis. 

2.  Survey  question(s)  used  to  measure  response. 

3.  A  statement  of  the  findings  relative  to  the 
question(s)  presented  in  four  ways: 

a.  Raw  scores  (in  tabular  form),  if  applicable. 

•  b.  Descriptive  form  (graphically). 
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c.  Mean  score,  and  z-value;  or  chi-squared  and 
chi-squared  (alpha)  scores. 

d.  Statement  of  the  acceptance  or  rejection  of  the 
null  hypothesis — based  upon  the  z-value  calculated  with 
respect  to  an  4  =  0.10  level,  and  further  clarified  in 
accordance  with  the  numeric  and  contextual  ratings  shown  in 
Table  II. 

4.  A  general  discussion  of  the  comments  made  by 
respondents  in  order  to  substantiate  the  mathematical 
analysis. 

Following  the  analysis  of  all  null  hypotheses  for  a 
particular  proposition,  a  conclusions  section  will  be  used 
to  address  the  issue  of  whether  or  not  the  criterion  test 
designated  for  that  proposition  was  satisfied  to  the  degree 
required  to  reject  it  successfully.  Reference  to  a  table 
containing  the  comments  of  respondents  on  different 
questions  is  also  provided. 

Analysis 

Table  IV  contains  the  results  of  all  hypothesis  testing 
conducted  in  this  research  project.  It  includes  comparisons 
of  means,  medians,  and  modes,  the  z- ,  za  and  p-values  for 
each  null  hypothesis  tested,  or  a  X2  and  a  Xa  if 
applicable;  and  a  "REJECT"  or  "DO  NOT  REJECT"  statement, 
with  respect  to  the  null  hypothesis,  as  applicable. 
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BY  PROGRAM  OFFICE  TYPE 


Figure  5:  Respondent  Demographics  III 
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Figure  6:  Respondent  Demographics  IV 


Figure  7:  Respondent  Demographics  v 


R&M  EDUCATION  AND  TRAINING 


Figure  8:  Respondent  Demographics  VI 


Figure  5: 


Respondent  Demographics  VII 


Proposition  1_.  The  tools  of  R&M  management  addressed 
in  this  research  study  have  had  little  or  no  effect  on 
achieving  R&M  requirements  in  ASD  program  offices  (Null 
Hypotheses  1  through  4). 

Null  Hypothesis  1_.  R&M  management  tools  associated 
with  the  planning  function  have  had  little  or  no  effect  on 
achieving  R&M  requirements  in  ASD  program  offices  (survey 
questions  la.  through  lg.). 

1 .  Findings  . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z -value:  x  =  4.026,  z  =  22.032 

c.  Decision:  Reject  the  null  hypothesis 


TABLE  IV 


Null 
Hyp  # 

Mean 

Median 

Mode 

Std 

Oev 

Z  - 

value 

z«  - 

value 

P  “ 
value 

X2- 

value 

X2- 

value 

Reject 

Null? 

1 

4.026 

4 

4 

1.827 

22.032 

1.28 

0.000 

— 

— 

REJECT 

2 

4.444 

5 

5 

1.793 

14.254 

1.28 

0.000 

— 

- - 

REJECT 

3 

4.188 

6 

4 

1.809 

17.874 

1.28 

0.000 

- - 

— 

REJECT 

4 

4.384 

5 

5 

1.929 

25.800 

1.28 

0.000 

— 

— 

REJECT 

5 

4.644 

5 

5 

1.944 

9.987 

1.28 

0.000 

— 

- - 

REJECT 

6 

3.933 

4 

4 

1.947 

7.522 

1.28 

0.000 

— 

— 

REJECT 

7 

4.267 

5 

5 

2.071 

8.150 

1.28 

0.000 

— 

— 

REJECT 

a 

4.667 

5 

5 

2.067 

9.465 

1.28 

0.000 

- - 

— 

REJECT 

9 

4.022 

4 

4 

1.983 

7.688 

1.28 

0.000 

— 

— 

REJECT 

10 

3.667 

4 

4 

1.989 

6.466 

1.28 

0.000 

- - 

— 

REJECT 

n 

3.778 

4 

4 

1.987 

6.845 

1.28 

0.000 

- - 

- - 

REJECT 

12 

4.5S6 

5 

5 

2.170 

8.675 

1.28 

0.000 

— 

— 

REJECT 
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Figure  10: 
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2.  Comments .  Using  the  scale  presented  in  Table 
II,  the  respondents  indicated  that  R&M  management  tools  had 
a  moderate  effect  on  the  planning  function  with  a  mean  score 
of  4.06.  The  three  documents  which  were  rated  highest  in 
effectiveness  by  respondents  were  the  Statement  of  Work 
(SOW),  Statement  of  Operational  Capability  (SOC),  and  the 
Instructions  to  Offerers.  However,  approximately  56%  of  the 
responses  in  this  area  were  rated  four  or  lower  on  the 
Lickert  scale.  There  were  viable  reasons  for  this.  First, 
a  large  group  of . respondents  claimed  no  knowledge  of  many  of 
these  documents.  Also,  many  rated  a  document  1  or  2  if  it 
was  no  longer  emphasized  in  the  present  stage  of  their 
program's  acquisition  cycle.  Finally,  there  were  reports  of 
several  of  these  documents  either  not  existing  or  being  not 
applicable  to  programs.  The  document  receiving  the  greatest 
number  of  low  ratings  was  the  MAJCOM  R&M  Plan. 

Null  Hypothesis  2.  R&M  management  tools  associated 
with  the  organizing  function  have  had  little  or  no  effect  on 
achieving  R&M  requirements  in  ASD  program  offices  (survey 
questions  lh.  and  lm.). 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 


b.  Mean  and  z-value:  x  =  4.444,  z  =  14.254 

c.  Decision:  Reject  the  null  hypothesis 


2.  Comments .  Of  the  ten  respondents  who  did 
supply  an  explanation  for  their  particular  responses  to  the 
questions  associated  with  this  hypothesis,  most  explanations 
were  negative.  Examples  include  a  Deputy  for  Engineering 
with  18  years  acquisition  experience  who  felt  SPO  R&M 
organizations  had  too  few  people  to  be  effective;  and  a  DPML 
with  ten  years  in  acquisition  who  stated  his  SPO's  R&M 
organization  did  not  provide  the  management  support  needed 
early  enough  in  the  design  stages,  forcing  the  program 
office  to  now  "scramble  to  recover."  There  were  positive 
responses,  however.  Examples,  here,  include  the  comment 
made  by  a  Deputy  for  Engineering  with  24  years  experience 
who  stated  that  in  his  SPO  the  R&M  organization  is  charged 
with  keeping  the  designers  on  top  of  the  R&M  issue.  Also,  a 
Deputy  for  Test  and  Evaluation  lauded  his  program  office's 
R&M  organization  for  their  daily  monitoring  of  R&M  issues, 
which  has  resulted  in  a  90%  mission  capable  rate. 

Null  Hypothesis  2*  R&M  management  tools  associated 
with  the  directing  function  have  had  little  or  no  effect  on 
achieving  R&M  requirements  in  ASD  program  offices  (survey 
questions  li.,  11.,  In.,  and  lo.). 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  4.188,  z  =  17.874 

c.  Decision:  Reject  the  null  hypothesis 
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2.  Comments .  Strong  support  was  provided  for  all 
but  one  of  the  directing  tools  surveyed.  Question  lo,  which 
measured  the  timing  of  R&M  activities,  had  the  weakest 
support.  Comments  here  ranged  from,  "Performance  and 
schedule  drove  everything,"  to,  "too  much  design  detail 
emphasis  [during]  Demonstration/Validation .  [This]  Needs  to 
be  more  focused  during  Full  Scale  Development."  Yet  one 
program  manager  rated  the  effectiveness  of  the  timing  of  R&M 
activities  at  6  stating,  "the  earlier,  the  better." 

Null  Hypothesis  4^.  R&M  management  tools  associated 
with  the  controlling  function  have  had  little  or  no  effect 
on  achieving  R&M  requirements  in  ASD  program  offices  (survey 
questions  lj.,  Ik.,  and  lp.  through  lu.). 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  ?  =  4.384,  z  =  25.800 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Strong  support  was  provided  for  all 
controlling  tools  surveyed.  Particularly  noteworthy  was  the 
effectiveness  of  incentives  and  warranties,  reliability 
demonstrations,  and  environmental  stress  screening  in 
achieving  program  R&M  requirements.  Also  critiqued  was  the 
author's  oversight  in  failing  to  obtain  a  measure  of  the 
effectiveness  of  maintainability  demonstrations  in  achieving 
program  R&M  requirements. 
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Conclusions .  For  Proposition  1  to  oe  rejected, 
three  of  the  four  associated  null  hypotheses  must  be 
rejected.  Since  this  is  the  case,  sufficient  evidence  is 
provided  to  infer  that  tools  of  R&M  management  addressed  in 
this  research  are  effective  in  achieving  R&M  requirements  in 
ASD  program  offices.  Based  on  the  mean  score  intervals 
outlined  in  Table  II,  a  moderate  degree  of  effectiveness  is 
attributed  to  the  R&M  management  tools  addressed  in  this 
research.  Table  IX  lists  all  respondent  verbal  comments 
provided  with  answers  to  survey  question  one. 

Proposition  2.  Program-specific  R&M  requirements  have 
had  little  or  no  influence  on  the  management  of  integrated 
logistics  support  (Null  Hypotheses  5  through  14). 

Null  Hypothesis  Program-specific  R&M 

requirements  have  had  little  or  no  influence  on  management 
of  maintenance  planning  (survey  question  2a.). 

1 .  Findings . 

a.  Test  Results:  TaDle  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  4.644,  z  =  9.987 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Sixty-nine  percent  of  respondents 
rated  maintenance  planning  at  a  level  of  5  or  higher  in  how 
much  its  management  is  influenced  by  program-specific  R&M 
requirements.  One  DPML  commented  that  maintenance  support 
concepts  are,  "key  to  developing  all  the  ILS  elements." 
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This  was  supported  by  another  DPML  who  said  it,  “forms  the 
basis  for  decisions  regarding  repair  level,  spares,  and 
maintenance  training." 

Null  Hypothesis  6.  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  manpower  and  personnel  (survey  question  2b.). 

1.  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  3.933,  z  =  7.522 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  In  this  instance,  58%  of  all 
responses  fell  below  a  rating  of  4  resulting  in  the  low  mean 
value  and  mid-range  z-value,  when  compared  to  the  other 
hypotheses.  One  Deputy  for  Engineering  commented  that, 
"[The]  User  estimates  his  manpower  level  high  no  matter  how 
stringent  the  R&M  requirements  are  set."  It  was  the  opinion 
of  one  DPML  that,  "ILS  is  not  a  viable  program..."  The  lack 
of  any  additional  meaningful  respondent  comments  makes 
further  analysis  of  these  results  difficult  without  a 
follow-up  study  designed  to  readdress  this  area. 

Null  Hypothesis  7_.  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  supply  support  (survey  question  2c.). 


1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  4.267,  z  =  8.150 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Six  respondents  provided  verbal 
explanations  of  their  ratings  in  this  question.  Those 
comments  are  split  50-50  with  respect  to  the  influence  of 
program-specific  R&M  requirements  on  the  management  of  this 
ILS  element.  The  small  number  of  meaningful  comments  also 
makes  analysis  of  these  results,  beyond  the  hypothesis  test, 
difficult  without  follow-up  research. 

Null  Hypothesis  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  support  equipment  (survey  question  2d.). 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  4.667,  z  =  9.465 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Sixty-five  percent  of  respondents 
felt  program-specific  R&M  requirements  earned  a  rating  of 
five  or  higher  in  their  influence  on  the  management  of 
support  equipment  issues.  One  Deputy  for  Test  and 
Evaluation  noted  that  a  high  degree  of  weapon  system 
reliability  could  lead  to  elimination  of  a  level  of  repair, 
thereby  impacting  system  support  equipment  requirements. 
Since  only  six  respondents  provided  comments  with  their 
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ratings,  further  explanation  of  test  results  were  difficult 
to  ascertain.  A  follow-up  study  appears  necessary. 


Null  Hypothesis  j?.  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  technical  data  (survey  question  2e.). 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  4.022,  z  =  7.688 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Only  three  respondents  chose  to 
provide  a  verbal  explanation  for  their  rating  in  this 
instance.  Two  of  the  responses  provided  accompanied  ratings 
of  seven.  The  lack  of  accompanying  respondent  comments 
makes  analysis  beyond  the  hypothesis  test  and  clarif ication 
of  the  results  difficult  without  follow-up  research. 


Null  Hypothesis  10.  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  computer  resources  (survey  question  2f.). 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 
•  b.  Mean  and  z-value:  x  =  3.667,  z  *  6.466 
c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  This  question  had  the  lowest  overall 
z-value  of  all  survey  questions.  Only  five  respondents 
provided  verbal  support  for  their  answers  to  this  question. 
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Although  it  was  the  intent  of  this  question  to  identify 
examples  of  the  influence  of  computer  resources  on  the 
management  of  ILS,  a  most  interesting  response  came  from  a 
DPML  who  approached  the  question  from  the  computer  resource 
acquisition  angle.  He  stated  that  although  computer 
resources  accounted  for  only  five  percent  of  his  program 
office's  total  acquisition  effort,  it  had  one  of  the  biggest 
impacts  on  schedule.  The  author's  personal  experience  with 
an  ongoing  cruise  missile  acquisition  program  can 
substantiate  this  comment--for  the  ability  to  deliver 
working  support  software  on  time  is  very  difficult.  Quite 
often,  because  of  the  level  of  sophistication  built-in  to 
weapon  system  computer  programs,  contractor  laboratory 
testing  of  an  updated  version  of  support  software  uncovers  a 
lack  of  ability  to  perform  intended  functions/enhancements 
without  degrading  previously  operable  functions. 

Null  Hypothesis  11 .  Program-specif ic  R&M 
requirements  have  had  little  or  no  influence  on  management 
.  of  training  (survey  question  2g . ) . 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  3.778,  z  =  6.845 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Sixty-eight  percent  of  the  ratings 
made  on  this  question  fell  at  or  below  four  on  the  Lickert 
scale.  As  with  the  previous  two  questions,  only  three 
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verbal  responses  were  obtained,  two  of  which  gave  ratings  of 
six  and  seven.  Analysis  of  responses  to  this  question 
rendered  the  second  lowest  z-value  on  the  survey.  As 
with  the  previous  two  questions,  very  few  verbal  responses 
were  provided  making  an  assessment  of  the  overall  outcome  of 
this  question  difficult. 

Null  Hypothesis  12 .  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  design  interface  (survey  question  2h.). 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  4.556,  z  =  8.675 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  This  question  received  more  ratings 
of  six  and  seven  than  any  other.  One  Deputy  for  Engineering 
with  24  years  acquisition  experience  called  design 
interface,  "the  single  biggest  impact  in  assuring  R&M  is 
built  in  [to  a  weapon  system] — not  added  on."  Interestingly, 
a  program  manager  with  10  years  experience  who  rated  the 
influence  R&M  requirements  in  his  program  had  on  design 
interface  at  two,  stated,  "it  should  be  at  7." 

Null  Hypothesis  13 .  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  facilities  (survey  question  2i.). 
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1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  3.733,  z  =  7.385 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Overwhelming  support  for  the  degree 
of  influence  R&M  requirements  have  on  management  of 
facilities  is  shown  by  the  fact  that  69%  of  respondents 
rated  the  degree. of  influence  at  four  or  higher.  A  Deputy 
for  Test  and  Evaluation  noted  that,  "R&M  suffers  severely  if 
facilities  are  lacking."  As  with  three  of  the  previous 
questions,  there  was  a  lack  of  verbal  responses  to  this 
question  as  well.  Five  respondents  chose  to  supply  verbal 
explanations  of  their  choice  of  rating.  In  response  to  the 
bimodality  of  rankings  obtained  in  this  question,  the  author 
attempted  to  ascertain  if  perhaps  there  was  a  difference  in 
preference  shown  by  respondents  assigned  to  aircraft  program 
offices  vice  missile  and  engine  program  offices.  In  other 
words,  does  one  of  these  groups  find  R&M  requirements  for 
the  management  of  facilities  acquisition  more  influential 
than  the  other?  The  rationale  for  this  is  that  there  may 
have  been  more  critical  factors  to  consider  when  acquiring 
support  facilities  for  one  versus  the  other.  Results 
indicate  that  there  is  no  appreciable  difference  ir  rating 
between  respondents  assigned  to  aircraft  program  offices 
vice  missile  and  engine  program  offices.  Therefore,  the 
cause  of  the  bimodality  cannot  be  readily  ascertained 
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without  additional  hypothesis  testing.  But  it  is  the 
opinion  of  the  author  that  this  occurrance  of  bimodality  is 
not  significant  for  three  reasons.  First,  it  is  the  only 
occurrance  of  bimodality  in  the  entire  survey.  Second,  the 
values  are  consecutive,  not  separated  by  other  rankings  on 
the  Lickert  scale.  And  third,  the  values  are  similar  to  all 
other  modal  values  obtained  from  all  Lickert  scale  survey 
questions.  However,  a  smaller  standard  deviation  and 
student-t  value  occurred  in  analysis  of  responses  from  the 
missile/engine  sample  subgroup  than  with  aircraft  program 
office  respondents.  This  would  indicate  less  variability  in 
the  responses  of  the  missile/engine  subgroup  than  in  those 
of  the  aircraft  subgroup,  although  the  difference  is  not 
significant.  Table  V  compares  the  means,  standard 
deviations,  and  student-t  values  from  both  sample  subgroups. 


TABLE  V 

RATING  COMPARISON  OF  AIRCRAFT  vs.  MISSILE/ENGINE  SUBGROUPS 

(FACILITIES  ISSUE) 


Aircraft 

Missile/Engine 

Subgroup 

Subqroup 

Mean 

3.696 

3.647 

Standard  Deviation 

2.055 

1.658 

Student-t  Value 

4.541 

4.716 

Null  Hypothesis  14 .  Program-specific  R&M 
requirements  have  had  little  or  no  influence  on  management 
of  packaging,  handling,  storage,  and  transportation  (survey 
question  2 j . ) . 
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1. 


Findings . 


a.  Test  Results:  Table  IV  and  Figure  10 

b.  Mean  and  z-value:  x  =  3.711,  z  =  7.241 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Seventy-three  percent  of  respondents 
rated  the  influence  of  program  R&M  requirements  on  the 
management  of  packaging,  handling,  storage  and 
transportation  issues  at  four  or  higher.  Unfortunately, 
here  again  only  three  respondents  chose  to  supply  verbal 
explanations  of  their  choice  of  rating  making  it  impossible 
to  completely  understand  the  overall  rating.  As  with  the 
previous  hypothesis,  the  author  wished  to  ascertain  if  in 
this  instance  a  difference  in  overall  rating  would  occur  if 
the  ratings  of  aircraft  program  office  respondents  were 
again  compared  to  those  of  missile/engine  program  office 
respondents.  The  rationale  for  making  this  comparison  was 
that  packaging,  handling,  storage,  and  transportation  issues 
may  perhaps  play  a  more  crucial  role  in  missile/engine 
acquisition  management  than  in  the  same  effort  for  aircraft; 
since  aircraft  effectively  transport  themselves  from  factory 
to  flightline,  but  missiles  and  engines  must  be  packaged  for 
transport  via  a  myriad  of  possible  modes  then  are  stored  by 
the  user  for  indefinite  periods  of  times.  Results  indicate 
that  here  again  there  was  no  appreciable  difference  in 
rating  between  respondents  assigned  to  aircraft  program 
offices  vice  missile  and  engine  program  offices.  As  with 


the  previous  comparison,  a  smaller  standard  deviation  and 
student-t  value  occurred  in  analysis  of  responses  from  the 
missile/engine  sample  subgroup  than  with  aircraft  program 
office  respondents.  This  would  indicate  less  variability  in 
the  responses  of  the  missile/engine  subgroup  than  in  those 
of  the  aircraft  subgroup,  although  again  the  difference  is 
not  significant.  Table  VI  compares  the  means,  standard 
deviations,  and  student-t  values  from  both  sample  subgroups. 

TABLE  VI 


RATING -COMPARISON  OF  AIRCRAFT  vs.  MISSILE/ENGINE  SUBGROUPS 
(PACKAGING,  HANDLING,  STORAGE  AND  TRANSPORTATION  ISSUE) 


Aircraft 

Missile/Engine 

Subqroup 

Subgroup 

Mean 

3.652 

3.765 

Standard  Deviation 

2.146 

1.641 

Student-t  Value 

4.251 

5.061 

Conclusions .  For  Proposition  2  to  be  rejected, 
six  of  the  ten  associated  hypotheses  must  be  rejected. 

Based  upon  the  responses  provided,  the  criteria  is  met. 
Therefore,  sufficient  evidence  is  provided  to  infer  that 
program-specific  R&M  requirements  do  influence  the 
management  of  integrated  logistics  support.  Based  upon  the 
mean  intervals  outlined  in  Table  II,  it  is  a  moderate  degree 
of  influence.  With  respect  to  the  ILS  elements  of 
facilities  and  packaging,  handling,  storage  and 
transportation,  it  was  shown  that  there  is  was  no 
appreciaole  difference  in  the  level  of  influence  program  R&M 
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requirements  had  on  their  management  whether  these  elements 
were  being  acquired  in  support  of  aircraft,  missiles,  or 
engines.  One  final  note  of  interest  was  that  with  each  of 
the  survey  questions  asked  in  support  of  this  proposition, 
there  were  four  to  five  respondents  who  consistently 
indicated  either  a  lack  of  familiarity  with  ILS  elements, 
the  opinion  that  ILS  management  was  not  part  of  their 
particular  duties,  or  that  management  of  integrated 
logistics  support  was  not  important  on  their  program.  A 
list  of  all  respondent  comments  to  questions  associated  with 
this  proposition  can  be  found  in  Table  X. 

Null  Hypothesis  15 .  There  is  no  difference  in  the  rank 
order  ASD  senior  level  managers  and  HQ  USAF  leadership  give 
to  the  goals  of  the  R&M  2000  Process  (survey  question  3). 

1.  Findings .  Table  VII  shows  the  observed 
proportion  of  respondents  preferring  option  a.  (top  of 
table)  from  each  paired  group  to  option  b.  (side  of  table). 
Table  VIII  shows  the  z-values  related  to  the  preference 
proportions  of  Table  VII.  Figure  11  depicts  the  interval 
scale  derived  using  Thurstone's  Case  V  model.  Figure  11 
highlights  the  rank  ordering  of  the  goals  of  the  Air  Force 
R&M  2000  Process  as  determined  by  survey  respondents. 

2.  Comments .  As  indicated  in  Figure  11, 
"increasing  combat  capability"  clearly  ranked  first  among 
respondents.  However,  beyond  this  initial  agreement  with 
what  is  outlined  in  the  R&M  2000  Process  pamphlet,  the 
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remainder  of  the  goals  of  R&M  2000  were  given  a  far 
different  order  of  importance. 

Ranked  second  by  respondents  was  "decrease  costs."  The 
small  difference  in  interval  between  these  first  two 
rankings  suggests  a  high  degree  of  agreed  preference  among 
respondents  for  seeing  these  two  goals  in  this  order. 
Additionally,  the  large  interval  between  the  first  two  and 
the  last  three  rankings  indicates  that  there  is  a  much 
stronger  preference  for  the  first  two  goals  than  for  the 
last  three,  with  "decreasing  mobility  requirements  per  unit" 
being  preferred  least. 

Conclusions .  For  null  hypothesis  15  to  be 
rejected,  no  difference  must  exist  in  the  way  HQ  USAF  and 
ASD  senior  managers  rank  the  goals  of  R&M  2000.  But,  as 
indicated,  the  rank  order  determined  by  ASD  senior  managers 
differs  from  that  outlined  in  HQ  USAF's  R&M  2000  Process 
pamphlet,  and  does  not  provide  support  for  the  hypothesis. 
Since  no  verbal  clarification  of  why  particular  preferences 
were  chosen  in  the  paired  difference  test,  the  question 
remains  as  to  why  such  a  large  difference  in  the  ordering  of 
these  goals  exists  between  the  ASD  senior  managers'  ranking 
and  that  of  HQ  USAF.  The  author  offers  two  possible 
reasons.  First,  there  could  exist  a  lack  of  knowledge  of 
the  order  of  importance  given  to  these  goals  by  HQ  USAF 
stemming  from  unfamiliar ity  with  the  concepts  of  the  R&M 
2000  process.  Or,  perhaps  legitimately,  there  is  a  more 
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realistic  ranking  based  upon  the  demands  of  the  existing 
acquisition  process,  which  cannot  be  avoided  lest  progress 
be  hampered  considerably.  This  research  effort  did  not 
address  this  issue. 


TABLE  VII 


PAIRED  COMPARISON  TEST  OBSERVED  PROPORTIONS 


Preferred 

Goal 

GOAL 

1 

2 

3 

4 

5 

1 

0.5000 

0.0698 

0.1429 

0.0909 

0.3409 

2 

0.9302 

0.5000 

0.2558 

0.4545 

0.6047 

3 

0.8571 

0.7442 

0.5000 

0.6279 

0.7500 

4 

0.9091 

0.5455 

0.3721 

0.5000 

0.8636 

5 

0.6591 

0.3953 

0.2500 

0.1364 

0.5000 

How  to  Read:  (In 

col.  1)  Ninety-three  percent 

of  the 

respondents  chose 

Goal  1  (Increase 

Combat  Capability) 

over  Goal  2  (Increase  Survivability 

. . . )  in  a 

paired 

comparison.  And  conversely 

(In  col 

.  2),  seven 

percent 

|  of  the  respondents 

preferred 

Goal  2 

to  Goal  1. 

TABLE 

VIII 

PAIRED  COMPARISON  TEST 

Z-VALUE  AND  R*  TABLE 

Preferred 

Goal 

GOAL 

1 

2 

3 

4 

5 

1 

0.00 

-1.47 

-1.06 

-1.33 

-0.40 

2 

1.47 

0.00 

-0.65 

-0.11 

0.26 

3 

1.06 

0.65 

0.00 

0.32 

0.67 

4 

1.33 

0.11 

-0.32 

0.00 

1.09 

5 

0.40 

-0.26 

-0.67 

-1.09 

0.00 

TOTAL 

:  4.260 

-0.970 

-2.700 

-2.210 

1.620 

Mean  (2) 

0.852 

-0.194 

-0.540 

-0.442 

0.324 

R* 

1.392 

0.346 

0.000 

0.098 

0.864 

RANK 

1 

3 

5 

4 

2 

_ 1 

HQ  USAF  RANKING 

RESPONDENT  RANKING 

Increase  Combat  -> 

-  1.39  - 

<-  Increase  Combat 

Capability 

Capability 

Increase  Survivability 

of  the  Combat  -> 

-  0.98  - 

Support  Structure 

-  0.86  - 

<-  Decrease  Costs 

Decrease  Mobility 

Rqmnts  Per  Unit  -> 

-  0.65  - 

Decrease  Manpower 

-  0.35  - 

<-  Increase  Survivability 

Rqmnts  Per  Unit  of  -> 

-  0.33  - 

Output 

-  0.10  - 

<-  Decrease  Manpower 

Rqmnts 

Decrease  Costs  -> 

-  0.00  - 

<-  Decrease  Mobility 

Rqmnts 

Figure  11:  Paired  Comparison  Test  Resultant 

Interval  Scale 


Proposition  .3.  An  adequate  level  of  R&M  education  and 
training  is  not  available  within  ASD  program  offices 
(Null  Hypotheses  16  and  17). 

Null  Hypothesis  16 .  An  adequate  supply  of  skilled 
and  experienced  R&M  personnel  does  not  exist  within  ASD 
program  offices  (survey  question  6). 

1 .  Findings  . 

a.  Test  Results:  Table  IV  and  Figure  12 

b.  Cni-squared  score:  X2=  2.81,  Xa=  2.71 

c . 


hypothesis 


Decision:  Do  not  reject  the  null 


2.  Comments.  Based  on  the  results  of  the  Chi- 
squared  test,  the  preferred  answer  to  this  question  was 
"No."  Fifty-nine  percent  of  respondents  provided  a  verbal 
explanation  of  their  choice  of  response.  Of  this 
percentage,  74%  were  to  the  negative.  The  majority  of  these 
responses  focused  on  the  perceived  problem  of  not  enough 
manpower  and/or  overworked  R&M  engineers  who  spend  more  time 
documenting  discrepancies  than  they  do  performing  actual 
engineering  work.  Still  other  respondents  commented  on  the 
lack  of  quality  R&M  personnel  available?  stating  that  the 
R&M  career  area  doesn't  draw  the  best  people  and  turnover  is 
too  quick. 


Satisfied  With  SPO  Level  of  R&M 
Education,  Training,  and  Expertise? 


No  Response  7* 
3 


Figure  12:  Survey  Question  Six  Answer  Breakdown 
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Null  Hypothesis  17 .  An  immediate  need  for  R&M 
education  and  training  exists  within  ASD  program  offices 
(survey  question  7). 

1.  Findings . 

a.  Test  Results:  Table  IV  and  Figure  13 

b.  Chi-squared  score:  X2 =  9.40,  Xa  =  6*25 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Based  upon  the  results  of  the  Chi- 
squared  test,  the  answer  preferred  by  respondents  was  "B," 
or  that  there  is  a  near  term  need~-within  a  year  or  so — for 
R&M  education  and  training. 


SPO  Immediacy  of  Need  for 
R&M  Education  and  Training 


Long  Term  I3R  None  11% 

8  5 


Figure  13:  Survey  Question  Seven  Answer  Breakdown 

Conclusions .  For  Proposition  3  to  be  rejected, 
both  null  hypotheses  16  and  17  must  be  rejected.  Results 
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indicate  that  although  senior  level  managers  feel  there  is 
no  immediate  need  for  more  R&M  education  and  training,  they 
did  feel  the  supply  of  skilled  and  experienced  R&M  personnel 
was  lacking.  Therefore,  there  is  insufficient  evidence  to 
infer  that  an  adequate  level  of  R&M  education  and  training 
exists  in  ASD  program  offices.  Survey  questions  four  and 
five  asked  respondents  to  indicate  those  methods  of 
education  and  training  now  employed  in  their  program 
offices,  as  well  as  those  which  they  felt  were  needed. 

Figure  14  graphically  portrays  the  responses  to  these 
questions.  As  is  indicated,  the  education  and  training 
tools  used  the  most  are  on-the-job  training  (#4/5D),  and 
symposia  and  mini-courses  (#4/5F).  Only  one  tool  was 
singled  out  as  not  presently  being  used — namely,  computer- 
aided  tools  and  training  (#4/51).  Interestingly,  it  was 
also  the  tool  selected  second  most  as  being  needed  as  an 
education  and  training  aid  (the  most  popular  being  in-house 
formal  training — #4/5E).  After  duty  hours  training  (#4/5G) 
was,  by  far,  the  least  favorite  tool  either  used  or  needed. 
The  author  believes  that  what  can  be  inferred  from  questions 
four,  five,  and  seven,  is  that  although  the  need  for 
education  and  training  is  not  immediate,  it  is  required;  and 
that  the  way  education  and  training  is  presently  being  done 
can  be  improved.  Table  XI  includes  a  listing  of  all 
comments  obtained  with  responses  to  question  six. 
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Education  &  Training  Tools 


NUHBER/TOOL 


ABCDEFCH!  J 
EDUCATION  Sc  TRAINING  TOOLS 


TOOL  STATUS 


■i  USED  (Qua  4)  I*  NEEDED  (Qum  5) 


3m  3u r»*f  Far  lluaui  o I  X-JUu  Latter* 


Figure  14:  Survey  Questions  Four  and  Five 

Answer  Breakdown 


Null 


is  18.  The  R&M  2000  process  has  had 


little  or  no  effect  on  the  management  of  R&M  in  A SD  program 
offices  (survey  question  8). 

1 .  Findings . 

a.  Test  Results:  Table  IV  and  Figures  10  &  15 

b.  Mean  and  z-value:  x  =  4.465,  z  =  9.742 

c.  Decision:  Reject  the  null  hypothesis 

2.  Comments .  Fifty-six  percent  of  responses  to 
this  question  were  rated  five  or  higher,  resulting  in  the 
large  z-value  and  strong  support  for  the  hypothesis.  Fifty- 
two  percent  of  the  respondents  provided  verbal  responses  to 


this  question.  Although  one-half  of  these  responses  spoke 
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favorably  of  R&M  2000,  there  were  some  interesting  comments 
offered  to  the  contrary.  One  Deputy  for  Test  and  Evaluation 
with  10  years  acquisition  experience  stated  he  was 
unfamiliar  with  the  R&M  2000  Process.  A  Deputy  for 
Engineering  with  28  years  experience  commented  that,  "no 
training/insight  [is  provided]  on  how  to  do  it."  A  DPML 
with  10  years  experience  called  the  R&M  2000  Process,  "just 
another  reporting  requirement."  One  Program  Director  with 
36  years  in  acquisition  stated,  "We  speak  to  R&M  2000  but 
show  little  progress  towards  achieving  its  objectives."  . 
Perhaps  the  feelings  of  these  respondents  was  best  summed  up 
by  the  Deputy  for  Engineering  with  28  years  experience  in 
acquisition  who  said,  "Concerned  personnel  plus  definitive 
design  requirements,  test  programs,  and  dollar  incentives 
make  an  R&M  program  effective." 

Conclusion .  For  null  hypothesis  18  to  be  rejected, 
the  mean  response  to  the  associated  question  must  be 
statistically  greater  than  1.75,  as  proven  by  the  resultant 
z-value  being  greater  than  1.28.  This  being  the  case, 
sufficient  evidence  exists  to  infer  that  the  Air  Force  R&M 
2000  Process  has  proven  moderately  effective  in  management 
of  R&M  requirements  in  ASD  program  offices.  Table  XII 
contains  the  comments  obtained  in  defense  of  ratings  given 
in  response  to  question  eight. 


Effectiveness  of  R&M  2000 
In  ASD  Program  Offices 


Figure  15:  Survey  Question  Eight  Answer  Breakdown 


Survey  Question  9.  In  your  opinion,  are  there  any 
particular  areas  which  need  more  emphasis  in  our  efforts  to 
improve  R&M  management  in  the  Air  Force? 

1.  Findings .  Figure  16 

2.  Comments .  Based  on  the  results  of  a  Chi- 
squared  test,  the  preferred  response  to  this  question  was 
"Yes."  Thirty-one  respondents  provided  a  verbal  explanation 
of  their  response  to  this  question.  Of  these,  29  were 
explanations  for  a  "Yes"  response.  Nine  respondents 
emphasized  the  need  for  more  education  and  training  in  the 
nuances  of  R&M  at  all  levels  of  program  management.  Three 
respondents  addressed  the  issue  of  needing  more  personnel 
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! 

and,  surprisingly,  only  one  respondent  addressed  the  money 

|  issue.  Other  responses,  as  well  as  portions  of  some  already 

highlighted,  offered  advice/recommendations  on  how  they  felt 

R&M  management  could  be  improved.  A  Deputy  for  Engineering 

with  30  years  acquisition  experience  suggests  that  we 

"train  managers  on  how  and  why  we  should  provide 
good  management  of  R&M  programs.  Don't  leave  [it] 
up  to  the  technical  organization." 

j  A  Program  Manager  with  20  years  experience  said, 

"People  of  all  backgrounds  need  to  know  the  basics  [of 

R&M]."  Another  Deputy  for  Engineering  with  28  years 

experience  suggested 

"closer  coordination  between  the  design  and 
manufacturing  functions  and  disciplines. 

Manufacturing  needs  to  get  involved  sooner  both 
at  ASD  and,  perhaps  more  importantly,  at  the 
contractor's  plant." 

A  Chief  Engineer  with  24  years  experience  commented 

"Contractor  high  level  management  has  to  take  on 
the  commitment  to  provide  reliable  and 
maintainable  hardware  at  the  beginning  of  a 
program. .. High  level  Air  Force  management  [has] 
to  get  this  message  to  industry .. .R&M  2000  has 
helped  to  accomplish  this  to  a  degree." 

,  Also  worthy  of  mention  are  the  two  "No"  responses.  The 

first  was  by  a  DPML  with  three  years  experience  who  said 

"People  assigned  to  the  R&M  program  would  be 
better  put  to  use  in  the  SPOs.  Let  the  program 
managers  work  R&M  as  required.  If  they  need  help, 
provide  training.  We  must  stop  this  stupid 
layered  management." 
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Figure  16:  Survey  Question  Nine  Answer  Breakdown 


The  other  was  by  a  Program  Manager  with  10  years 
experience  who  stated 

"Too  many  initiatives  all  at  once.  Everyone 
wants  to  jump  on  the  R&M  2000  bandwagon.  Let's 
take  some  time  to  see  how  the  next  generation  of 
weapon  systems  have  benefitted  before  our  next 
round  of  initiatives." 

A  complete  list  of  comments  make  by  respondents  can  be 
found  in  Table  XIII. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


Introduction 

This  chapter  presents  the  findings  of  this  research. 
Findings  are  presented  by  investigative  question  in  the 
order  in  which  they  were  introduced  in  Chapter  I.  A  summary 
section  follows,  in  which  a  final  conclusion  is  presented  to 
answer  the  research  question.  Following  presentation  of  all 
findings,  some  recommendations  for  further  study  of  this 
topic  are  presented. 

Question 

Question.  How  do  senior  ASD  managers  feel  about  the 
effectiveness  of  existing  R&M  management  tools  as  aids  to 
performing  functional  duties? 

Findings .  Based  upon  the  answers  to  survey  question  1, 
it  was  determined  that  the  R&M  management  tools  presented  in 
the  survey  were  moderately  effective  as  aids  to  performing 
the  functional  management  duties  of  planning,  organizing, 
directing,  and  controlling.  Survey  respondents  levied  their 
harshest  criticisms  on  the  virtual  non-existence  or  non-use 
of  MAJCOM  R&M  Plans.  Two  categories  were  rated  strong 
overall  with  respect  to  their  management  effectiveness. 

First  were  the  controlling  tools,  with  the  highest  ratings 
given  to  the  effectiveness  of  incentives  and  warranties, 
reliability  demonstrations,  ana  environmental  stress 
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screening.  Next  were  the  directing  tools--namely :  new 
applications  of  R&M  technology,  R&M  program  constraints  and 
trade-offs,  previous  R&M  lessons  learned,  and  the  timing  of 
R&M  activities  in  relation  to  other  program  activities. 
Tables  and  figures  comparing  and  contrasting  respondent 
ratings  to  all  categories  and  elements  measured  by  question 
one  can  be  found  in  Appendix  B. 

Question  2 

Question .  How  are  the  ten  integrated  logistics  support 
elements  affected  by  increased  emphasis  on  R&M? 

Findings .  Based  upon  the  answers  to  survey  question  2, 
it  was  determined  that  program-specific  R&M  requirements  had 
a  moderate  influence  on  the  management  of  integrated 
logistics  support.  The  ILS  element  rated  most  influenced  by 
program  R&M  requirements  was  design  interface  (mean  score  of 
4.667).  This  was  strongly  supported  by  the  respondents' 
verbal  comments  which  accompanied  some  of  the  ratings  (see 
Table  X).  Maintenance  planning  received  the  second  highest 
mean  score  (4.644)  with  respect  to  the  degree  of  influence 
program  R&M  requirements  had  on  its  management.  The  ILS 
element  rated  least  influenced  by  program  R&M  requirements 
was  training.  See  Appendix  B  for  a  comparison/contrasting 
of  respondent  ratings  for  all  10  ILS  elements. 
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Question  3_ 

Question .  What  priority  is  given  to  R&M  2000  goals  by 
senior  level  ASD  managers? 

Findings .  Based  upon  the  answers  to  survey  question  3, 
it  was  determined  that  ASD  senior  level  managers  placed  the 
goals  of  the  Air  Force  R&M  2000  process  in  a  different  order 
of  priority  than  was  outlined  by  HQ  USAF  in  their  R&M  2000 
Process  pamphlet.  Placing  the  goal  "Increase  Combat 
Capability"  first  was  the  only  source  of  agreement  between 
the  two  groups.  ASD  senior  managers  ranked  the  goal 
"Decrease  Costs"  second  in  order  of  priority,  in  contrast  to 
the  priority  it  is  given  by  HQ  USAF — last.  Because  there 
was  no  requirement  for  respondent  justification  of  choice  in 
each  paired  comparison  situation,  the  author  based  the 
findings  made  in  this  situation  on  the  assumption  that  given 
the  choices  outlined  in  each  paired  comparison  situation, 
respondents  chose  one  over  the  other  based  upon  their  years 
of  acquisition  management  experience  and/or  decisions  made 
in  similar  "real  life"  situations. 

Question  j4 

Question .  How  satisfied  are  senior  level  ASD  managers 
with  R&M  education  and  training,  and  R&M  expertise  within 
their  program  offices? 

Findings .  Based  upon  the  answers  to  survey  questions 
4,  5,  6,  and  7;  it  was  determined  that  ASD  senior  managers 
are  not  fully  satisfied  with  the  methods  of  R&M  education 


72 


and  training  utilized  within  their  program  offices,  nor  are 
they  fully  satisfied  with  the  amount  of  R&M  expertise  they 
have  resident  in  their  program  offices.  Many  different 
sources  of  R&M  education  and  training  are  being  exploited 
in  ASD  program  offices,  the  most  popular  being  on-the-job 
training,  and  use  of  symposia/mini-courses.  Respondents 
also  voiced  a  desire  to  see  increased  use  of  both  in-house 
formal  training  and  computer-aided  tools/training  as  means 
of  increasing  the  R&M  competence  of  program  office  assigned 
personnel.  With  respect  to  the  immediacy  of  need  for  these 
enhancements  to  SPO  R&M  education  and  training  activities, 
it  was  found  that  respondents  would  like  to  see  these 
enhancements  made  within  a  year.  This  position  is  supported 
by  the  respondents'  opinion  that  there  is  a  lack  of 
experienced  R&M  personnel  within  ASD  program  offices.  The 
vast  majority  of  verbal  responses  which  accompanied  answers 
to  question  6  substantiate  this  (see  Table  XI). 

Question  5^ 

Question .  How  effective  is  the  Air  Force  R&M  2000 
program  within  ASD  program  offices,  and  how  can  this 
effectiveness  be  improved? 

Findings .  Based  upon  the  answers  to  survey  question  8, 
it  was  determined  that  respondents  felt  the  Air  Force 
R&M  2000  program  was  moderately  effective  within  ASD  program 
offices.  Verbal  responses  which  accompanied  ratings  showed 
approximately  a  50-50  split  as  to  its  value  (see  Table  XII). 
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Question  9  queried  respondents  as  to  possible  additional 
improvements  in  the  way  R&M  is  managed.  Suggestions 
included  increasing  the  amount  of  R&M  education  and  training 
offered,  increasing  the  number  of  personnel  charged  with 
monitoring  R&M  activities,  improving  communication  between 
designers  and  manufacturers,  and  enforcing  active  contractor 
participation  in  R&M  improvement  efforts  (see  Table  XIII). 

Summary 

The  Research  Question .  What  are  the  attitudes  of 
senior  level  managers  in  ASD  regarding  the  utility  of  the 
Air  Force  R&M  2000  Program? 

Conclusion .  Based  upon  analysis  of  findings,  it  was 
determined  that  the  Air  Force  R&M  2000  Process  has  been 
moderately  useful  in  the  conduct  of  R&M  management.  Some  of 
the  R&M  2000  building  blocks  addressed  in  this  survey,  such 
as  reliability  demonstrations,  environmental  stress 
screening,  incentives  and  warranties — to  name  a  few,  were 
viewed  as  important  tools  in  the  fight  to  improve  the 
reliability  and  maintainability  of  newly  acquired  weapon 
systems  and  modifications.  There  was,  however,  a 
significant  disagreement  between  senior  level  ASD  managers 
and  HQ  USAF  personnel  as  to  which  goals  of  the  R&M  2000 
Process  should  have  priority  over  the  others.  Though  both 
groups  agreed  that  the  highest  priority  be  given  to 
increasing  combat  capability,  senior  level  managers  in  ASD 
felt  that  second  on  the  priority  l.Lst  should  be  decreasing 
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costs  (rated  the  last  priority  by  HQ  USAF),  vice  the  HQ  USAF 
number  two  priority  of  increasing  survivability  of  the 
combat  support  structure  (rated  third  by  senior  ASD 
managers ) . 

It  was  also  noted  by  the  author  that  there  was  a  degree 
of  non-familiarity  with  the  R&M  2000  Process  manifest  by 
several  of  the  respondents.  This  non-familiarity  was 
identified  in  two  ways.  The  first  was  through  comments  made 
on  various  surveys  which  stated  the  respondent  was  not 
familiar  with  the  concept.  The  second,  more  subtle,  clue 
was  uncovered  through  reading  respondent  comments  included 
with  questions.  .  The  best  example  was  Respondent  #20,  a  DPML 
whose  answer  to  question  9  infers  that  R&M  2000  is  a  program 
to  which  people  must  be  assigned,  further  drawing  down  the 
number  of  people  to  work  important  program  issues.  In  fact, 
this  same  individual  commented  that  integrated  logistics 
support  is  not  a  viable  program  when  working  in  the  arena  of 
modifications.  There  also  was  the  comment  made  by 
Respondent  #34,  a  Program  Director,  who  stated  he  does  not 
have  enough  people,  "to  work  any  such  program."  Another 
example  was  Respondent  #10,  a  Deputy  for  Engineering  who 
commented  that,  "some  policy  letters  have  been  received  and 
implemented."  Another  Deputy  for  Engineering  felt  R&M  2000 
was,  "too  difficult  to  implement  due  to  lack  of  experienced 
people."  Finally,  there  was  the  comment  made  by  a  DPML  that 
R&M  2000  does  not  get  enough,  "emphasis  and  support  from  the 


SPO. "  All  these  comments/  in  the  author's  assessment,  point 
to  a  lack  of  basic  understanding  of  what  the  R&M  2000 
Process  was  designed  for.  One  respondent  commented  he  had 
received  no  training/insight  into  how  to  do  it — the  pamphlet 
itself  provides  all  the  insight  one  needs  to  grasp  the 
concept.  Another  commented  his  program  office  dealt 
primarily  with  the  purchase  of  off-the-shelf  aircraft — but 
this  is  no  excuse  to  overlook  the  importance  of  increased 
emphasis  on  R&M.  In  summary,  although  the  R&M  2000  Process 
has  proved  moderately  effective  in  its  usefulness  as  an  R&M 
management  tool,  there  is  indication  of  the  existence  of  a 
certain  degree  of  non-familiarity  and/or  misinterpretation 
of  both  what  the  R&M  2000  Process  is  and  its  usefulness. 

Recommendations  For  Further  Study 

This  section  is  divided  into  two  areas  which  the  author 
deemed  important  to  emphasize — improvement  of  the  survey 
instrument  and  specific  areas  where  further  research  is 
required  for  comparing/contrasting  findings. 

Survey  Improvement .  The  author  feels  that  although  the 
survey  design  proved  adequate  for  the  exploratory  nature  of 
this  research  project,  there  are  several  deficiencies  in 
need  of  correction  before  the  instrument  should  be  used  in  a 
follow-on  study. 

Personal  Data .  The  author  failed  to  identify  the 
fact  that  there  would  be  a  large  number  of  Lieutenant 
Colonels,  Colonels,  and  GS-15  and  higher  individuals 
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surveyed,  given  the  level  of  management  personnel  this 
study  reached.  As  a  result,  a  less  than  adequate 
demographic  breakdown  of  respondents,  by  rank,  occurred. 

In  the  future,  care  should  be  taken  to  better  identify 
respondents  by  rank. 

Questions  1_  and  2.  The  author  was  disappointed  by 
the  low  number  of  verbal  responses  and  non-responses 
accompanying  ratings.  It  is  the  author's  opinion  that  a 
degree  of  misunderstanding  of  questions  may  have  been  the 
reason.  In  the  future,  it  may  prove  beneficial  to  include 
an  example  answer  so  that  respondents  have  an  idea  of  the 
direction  the  researcher  wishes  to  go  with  the  question. 

Question  1_.  Only  a  limited  number  of  the  R&M  2000 
building  blocks  were  measured  directly  in  the  survey.  As  a 
result,  it  was  not  possible  to  get  a  complete  measure  of 
respondent  familiarity  with  the  concepts  of  R&M  2000.  Since 
familiarity  with  R&M  2000  appears  to  be  a  potential  area  for 
future  study,  the  author  recommends  rephrasing  question  one 
so  that  all  R&M  2000  building  blocks  are  included  for  study 
in  the  research  instrument. 

Areas  For  Future  Study ■  As  was  stated  in  Chapter  I, 
there  are  four  other  product  divisions  in  Air  Force  Systems 
Command,  and  other  organizations  involved  in  weapon  system 
acquisition.  All  are  concerned  with  reliability  and 
maintainability  to  varying  degrees  in  varying  areas  of 
emphasis.  Use  of  the  survey  instrument  introduced  here, 
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after  the  recommended  changes  are  made,  to  measure  R&M 
management  behavior  in  each  of  these  organizations  would 
provide  a  wealth  of  information  useful  in  shaping  the  type 
of  R&M  education  and  training  needed  to  prepare  future 
logisticians  for  their  duties.  The  author  recommends, 
however,  that  this  study  first  be  repeated  in  ASD  in  order 
to  validate  the  updated  survey  instrument. 

The  author  believes  assessing  the  effectiveness  of  R&M 
management  and  the  familiarity  of  managers  with  the  tools  of 
R&M  management  in  each  of  these  organizations  is  critical  to 
ensuring  that  future  weapon  system  acquisition  endeavors  do 
not  repeat  the  errors  of  the  past.  It  should  be  the  goal  of 
the  acquisition  community  to  see  that  a  cadre  of  personnel 
is  spawned  with  an  appreciation  for  how  emphasis  on  R&M 
early  in  design  can  improve  our  combat  capability  and 
survivability  while  reducing  manpower  and  mobility 
requirements  and,  hopefully,  life  cycle  costs.  This  is  an 
avenue  of  research  which  should  not  be  abandoned. 
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APPENDIX  A 


COVER  LETTER  AND  MAIL  SURVEY 


TO 

ATT*  O#-. 


cs 


DEPARTMENT  OF  THE  AIR  FORCE 

HCAOOUAATKM  AC  AO  NAUTICAL  SYSTKMS  OIVISION  IAVSC1 
WMK3H T-AA rre ASCN  AIA  romcc  CASK.  OHIO  4SA33-SS03 


t  1  APR  1983 


mmucct:  R4M  Management  Survey  (Survey  Numoer  88-39) 


TO: 


1.  The  attached  survey  Is  part  of  an  AFIT  research  project  to  examine 
the  management  of  reliability  and  maintainability  in  the  acquisition 
process.  Tour  participation  in  this  research  endeavor  is  voluntary;  but, 
your  experience,  knowledge,  and  expertise  would  certainly  be  appreciated 
and  I  encourage  you  to  respond.  Individual  responses  will  be  combined 
with  others  and  not  be  attributed  to  you  personally.  For  further 
information,  contact  Capt  Ri buffo.  AFIT/LSfi,  extension  55435  or  his 
advisor,  Lt  Col  Materna.  AFIT/LSM,  extension  55023. 


2.  Please  fill  out  the  attached  questionnaire  and  return  in  the  enclosed 
enveloju^by  6J^T  1988. 


RONALO  H.  TRAUOT 
Colonel.  USAF 
Chief  of  Staff 


1  Atch 

Questionnaire 
w/return  envelope 


USAF  SCN  88-39 
Expires  31  May  88 

Instructions:  Select  only  ONE  answer  to  each  question  in  the 
survey,  except  where  instructions  indicate  otherwise.  Mark 
your  answers  on  the  survey  itself  by  simply  circling  the 
appropriate  letter/number.  Make  additional  comments  at  the 
areas  provided  at  the  specific  question. 


Personal  Data 

1.  What  is  your  present  rank? 


a. 

Major  or  below 

d. 

GS-9 

to 

GS-12 

b. 

Lt  Colonel  or  Colonel 

e . 

GS-13 

to 

GS-14 

c. 

General  Officer 

f . 

GS-15 

or 

higher 

2.  Time,  in  years,  you  have  been  involved  in  weapon  system 

acquisition:  _ 

3.  What  position  do  you  presently  hold? 


a. 

Program  Manager 

d. 

Deputy  for  T&E 

b. 

DPML 

e. 

ILS  Manager 

c. 

Deputy  for  Engineering 

f . 

Other 

4. 

Time,  in  years,  in  present  position: 

5. 

To 

which  type  program  office 

are 

you  currently  assigned? 

a. 

Tactical  Aircraft 

e . 

Strategic  Missile 

b. 

Strategic  Aircraft 

f . 

Tactical  Missile 

c . 

Airlift  Aircraft 

g. 

Tactical  Engine 

d. 

Training  Aircraft 

h . 

Other 

6. 

Have  you  received  any  formal 
education  or  training? 

R&M 

management  related 

a .  Yes 

b .  No 

7. 

If 

you  answered  "Yes"  to  question 

6,  please  qualify  your 

answer : 


DATE: 


COURSE: 


LOCATION: 


M*l 


i‘-d  16 l  I.JU-  l 


1.  Please  rate  the  degree  of  effectiveness  the  following 
areas  have  in  achieving  R&M  requirements  on  your  program: 


NOT 

EFFECTIVE 


VERY 

EFFECTIVE 


a.  Statement  of  Work 


Comments? 


b.  Statement  of  Need 


Comments? 


c.  Statement  of 

Operational  Capability 

Comments? 


d.  Requirements  Correlation 
Matrix 

Comments? 


e.  R&M  Management  Plan 


Comments? 


f.  MAJCOM  R&M  Plan 


Comments? 


g.  instructions  to  Offerers 


Comments? 


1  2  3  4  5  6  7 


1  2  3  4  5  6  7 


1  2  3  4  5  6  7 


1  2  3  4  5  6  7 


1  2  3  4  5  6  7 


1  2  3  4  5  6  7 


1  2  3  4  5  6  7 
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NOT 

EFFECTIVE 


VERY 

EFFECTIVE 


h.  SPO  R&M  Organization  1234567 

Comments?  _ 


i.  New  Applications  of 

R&M  Technology  1234567 

Comments? 


j.  Collection  and  Use  of 

Field  R&M  Data  1234567 

Comments? 


k.  R&M  Management  Information 

Systems  1  2  3  4  5  6  7 


NOT 

EFFECTIVE 


VERY 

EFFECTIVE 


o.  Timing  of  R&M  activities 
in  relation  to  other 

program  activities  1234567 

Comments?  _  _  _ 


1 


p.  Incentives  and  Warranties  1234567 

Comments? 


q.  Established  R&M 

Measures  of  Merit  1234567 

Comments? 


r.  R&M  Personnel  Management  1234567 

Comments? 


s.  Documented  Contractor  R&M 

Policies  and  Practices  1234567 

Comments? 


< 


i 

1 


t.  Reliability  Demonstrations  1234567 
Comments? 


u.  Environmental  Stress 
Screening 

Comments? 


FT'TZr 


Kt**.*' 


■ 


2.  Please  rate  the  degree  of  influence  program  R&M 
requirements  have  on  managing  each  integrated  logistics 
support  element; 


NO 

INFLUENCE 


SIGNIFICANT 

INFLUENCE 


a.  Maintenance  Planning 
Example?  _ 


1  2  3  4  5  6  7 


b.  Manpower  &  Personnel  1234567 

Example?  _ 


c.  Supply  Support 
Example?  _ 


1  2  3  4  5  6  7 


d.  Support  Equipment 
Example?  _ 


1  2  3  4  5  6  7 


e.  Technical  Data 
Example?  _ 


1  2  3  4.5  6  7 


f.  Computer  Resources 
Example?  _ 


1  2  3  4  5  6  7 


g.  Training 
Example? 


1  2  3  4  5  6  7 
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NO 

INFLUENCE 


SIGNIFICANT 

INFLUENCE 


h.  Design  Interface  1234567 

Example?  _ 


i.  Facilities  1234567 

Example?  _ _ _ 


j.  Packaging,  Handling, 

Storage,  and 

Transportation  1234567 

Example?  _ 


3.  Listed  below  are  pairs  of  possible  program  management 
objectives.  For  each  pair,  select  the  one  which  you  feel  is 
the  more  important  of  the  two  in  your  program.  Each 
objective  will  appear  more  than  once,  so  consider  each  pair 
independently . 

a.  increasing  combat  capability 

b.  increasing  survivability  of  the  combat  support  structure 

a.  increasing  survivability  of  the  combat  support  structure 

b.  decreasing  manpower  requirements  per  unit  of  output 

a.  decreasing  manpower  requirements  per  unit  of  output 

b.  decreasing  acquisition,  operation,  and  support  costs 

a.  increasing  combat  capability 

b.  decreasing  manpower  requirements  per  unit  of  output 

a.  decreasing  mobility  requirements  per  deploying  unit 

b.  decreasing  acquisition,  operation,  and  support  costs 

a.  increasing  survivability  of  the  combat  support  structure 

b.  decreasing  acquisition,  operation,  and  support  costs 

a.  increasing  combat  capability 

b.  decreasing  acquisition,  operation,  and  support  costs 
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a.  decreasing  mobility  requirements  per  deploying  unit 

b.  decreasing  manpower  requirements  per  unit  of  output 

a.  increasing  combat  capability 

b.  decreasing  mobility  requirements  per  deploying  unit 

a.  increasing  survivability  of  the  combat  support  structure 

b.  decreasing  mobility  requirements  per  deploying  unit 


4.  What  is/are  your  program  office's  current  method(s)  for 
satisfying  R&M  education  and  training  requirements?  Circle 
all  that  are  applicable. 

a.  None  g.  After  Duty  Hours  Training 

b.  Training  Manuals/Books  h.  College  Degree  Programs 

c.  Self-Study  Courses  i.  Computer-Aided  Tools 

d.  OJT  Training  and/or  Training 

e.  In-House  Formal  Training  j.  Other:  _ 

f.  Symposia/Mini-courses  _ 


5.  In  what  area(s)  of  R&M  education  and  training  does 
emphasis  need  to  be  placed  in  your  program  office?  Circle 
all  that  are  applicable. 

a .  None 

b.  Training  Manuals/Books 

c.  Self-Study  Courses 

d.  OJT  Training 

e.  In-House  Formal  Training 

f.  Symposia/Mini-courses 


6.  Based  upon  your  experience,  is  there  an  adequate  supply 
of  skilled  and  experienced  R&M  personnel  within  your  program 
office? 

a.  Yes  b.  No 


g.  After  Duty  Hours  Training 

h.  College  Degree  Programs 

i.  Computer-Aided  Tools 

and/or  Training 

j.  Other:  _ 


Please  Explain: 


7.  Please  indicate  the  immediacy  of  need  for  R&M  education 
and  training  in  your  program  office: 

a.  Immediate. 

b.  Near  Term  (within  the  year). 

c.  Long  Term  (one  to  five  years). 

d.  None 

8.  How  would  you  rate  the  effectiveness  of  the  Air  Force 
R&M  2000  Program  in  your  office? 


NOT 

VERY 

EFFECTIVE 

EFFECTIVE 

1  2 

3 

4 

5 

6  7 

Please  explain: 


9.  In  your  opinion,  are  there  any  particular  areas  which 
need  more  emphasis  in  our  efforts  to  improve  R&M  management 
in  the  Air  Force? 

a.  Yes  b.  No 


Please  explain: 


THANK  YOU  FOR  YOUR  COOPERATION  AND  ASSISTANCE! 
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APPENDIX  B 


VERBAL  RESPONSES  TO  SURVEY  QUESTIONS 


TABLE  IX 


VERBAL  RESPONSES  TO  QUESTION  1 


Ques . 
NO. 

Duty 

Title 

#  Yrs 
Acq. 

Rat¬ 

ing 

Response 

la. 

DPML 

10 

5 

Wasn't  put  together  very  well, 
but  definitely  the  contract 
leverage  driver  [is]  available. 

DPML 

3 

3 

Program  deals  primarily  with 
modifications.  R&M  pre- 
established  because  of  use  of 
off-the-shelf  components. 

PM 

10  + 

7 

If  it  isn't  included,  you  won't 
get  it. 

DTE 

1 

NR 

The  F-16  program  has  been  ongoing 
for  10+  years.  I'm  not  aware  of 
the  relationship  of  the  SOW  to 
current  R&M  requirements. 

OTHER 

8 

6 

Including  the  speci f ication 
requirements  within  the  contract 
( RFP  Package) . 

DPML 

16 

6 

Most  critical.  If  the  contractor 
is  not  properly  tasked  for  an 
effective  R&M  program,  you  won't 
have  one. 

PM 

10 

0 

I  do  not  deal  directly  with 
program  documentation  at  specific 
detail  level. 

lb. 

DEN 

24 

3 

Tends  to  be  too  detail  design 
oriented.  Should  be  focused  on 
what  is  desired  at  the  system 
level  and  not  tell  "how  to." 

DPML 

3 

2 

See  response  to  question  la. 

DTE 

10 

0 

No  knowledge. 

DPML 

7 

4 

Sometimes  watered  down  to  limit 
development  of  non-recurring 
costs . 

DPML 

8 

5 

Program  in  FSD .  SON  changes 
require  money  and  new  start. 

DTE 

1 

NR 

See  response  to  question  la. 

TABLE  IX  -  Continued 


VERBAL  RESPONSES  TO  QUESTION  1 


Ques . 
No. 

Duty 

Title 

#Yrs 

Acq. 

Rat 

ing 

Response 

lb. 

OTHER 

8 

5 

Only  effective  if  R&M  requirements 
are  included.  The  majority  of 

SONS  do  not  include  [them]. 

DPML 

16 

3 

O.K.  as  goes,  but  SONs  generally 
treat  R&M  requirements  as  separate 
values  with  little  accounting  for 
the  [relation]  between  them. 

PM 

10 

0 

See  response  to  question  la. 

lc. 

DPML 

8 

4 

See  response  to  question  lb. 

DTE 

1 

NR 

See  response  to  question  la. 

OTHER 

8 

5 

See  response  to  question  lb. 

DPML 

16 

3 

See  response  to  question  lb. 

PM 

10 

0 

See  response  to  question  la. 

Id. 

DTE 

2 

1 

Program  has  a  BCM ,  not  an  RCM . 

DPML 

10 

2 

Not  really  used  or  maintained  as 
it  should  be. 

DEN 

24 

5 

Can  be  effective  if  used  at  top 
level  [and  is]  not  detailed. 

DPML 

3 

2 

See  response  to  question  la. 

DTE 

1 

NR 

We  are  currently  flight  testing 
the  Block  40  version  on  the 

F-16C/D.  The  matrix  does  call  out 
goals  we  wish  to  achieve. 

PM 

10 

0 

See  response  to  question  la. 

le. 

DPML 

10 

2 

Not  used  as  a  reference  document 
as  it  should  be. 

DEN 

24 

5 

Plans  never  did  anything  but 
highlight  interests.  It  needs  to 
get  to  the  designer  to  make 
him/her  recognize  it  is  their  job, 
not  somebody  else's. 

DPML 

3 

2 

See  response  to  question  la. 

DTE 

10 

0 

See  response  to  question  lb. 

If. 

DTE 

2 

1 

Non-existent . 

DPML 

10 

1 

Not  aware  there  is  one. 

DEN 

24 

4 

See  response  to  question  le. 

DPML 

3 

1 

Not  used  in  modifications. 

DTE 

10 

0 

See  response  to  question  lb. 

TABLE  IX  -  Continued 


VERBAL  RESPONSES  TO  QUESTION  1 


Ques . 
No. 

Duty 

Title 

#Yrs 

Acq. 

Rat 

ing 

Response 

If. 

DTE 

1 

7 

We  have  exceeded  all  TAP  aircraft- 
related  standards  for  the  past 
several  years. 

DTE 

17 

1 

None. 

DEN 

29 

1 

Have  not  seen  [one]. 

lg. 

DEN 

30 

7 

Must  have  good  system  spec 
requirements,  and  tests  or 
demonstrations  [included]. 

DEN 

24 

7 

Showed  contractors  we  were 
serious . 

lg. 

DPML 

3 

1 

See  response  to  question  la. 

DPML 

16 

3 

Can  set  the  tone. 

lh. 

DPML 

10 

4 

Not  enough  management  support  in 
early  design  stages.  Now  it's  a 
scramble  to  recover. 

DEN 

18 

2 

Too  few  people. 

DEN 

24 

5 

Their  primary  purpose  is  to  assure 
the  design  side  of  the  house  keeps 
on  top  of  R&M. 

DPML 

3 

1 

See  response  to  question  la. 

DTE 

1 

7 

Through  daily  monitoring  at  the 

SPO  and  initiation  of  the  "Falcon 
90"  Program,  we  have  achieved,  and 
are  maintaining,  a  90%  MC  rate  on 
the  [F-16]C/D  force. 

PM 

10 

5 

Organizational  effectiveness  is  a 
direct  result  of  the  effectiveness 
of  the  people  charged  with  the 
responsibility  [to  manage  R&M]. 

li. 

DTE 

8 

2 

Applications  being  introduced  too 
late  to  be  meaningful. 

DPML 

10 

6 

Maintainability  of  low 
observables . 

DEN 

24 

3 

I  do  not  believe  there  is  such  a 
thing.  R&M  should  be  an  inherent 
part  of  any  new  technology  [that] 
is  inseparaole  from  the  "rest  of 
performance . " 

DPML 

3 

1 

See  response  to  question  la. 
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TABLE  IX  -  Continued 


VERBAL  RESPONSES  TO  QUESTION  1 


Ques . 
No . 

Duty 

Title 

#Yrs 

Acq. 

Rat¬ 

ing 

Response 

li . 

DPML 

7 

4 

Limited  by  real-time  budget 
constraints  despite  life  cycle 
cost  savings! 

- 

DTE 

1 

5 

We  are  currently  looking  at  radar 
updates ■ incorporating  R&M 
improvements . 

OTHER 

8 

6 

Very  effective  if  implemented 
adequately  [so]  as  to  meet 
specific  program  requirements  (ie, 
ASD's  Integrity  Program). 

PM 

10 

0 

Not  observed. 

lj. 

DPML 

10 

3 

Contractor  still  puts  too  much 
focus  on  pre-delivery  issues  at 
the  expense  of  post-delivery. 

DEN 

24 

1 

Generally,  it  is  a  tracking  of 
what  is  broke.  This  is  an  endemic 
problem  we  [will]  address  when  we 
start  FSD. 

DPML 

3 

1 

See  response  to  question  la. 

DTE 

10 

1 

Not  applicable  to  ATF . 

OTHER 

14 

3 

Not  much  yet. 

PM 

10  + 

7 

If  results  are  warranted. 

DTE 

1 

7 

We  collect  worldwide  data  weekly 
and  brief  the  SPO  Director.  We 
use  this  data  to  focus  on  problem 
systems  and  components  and  try  to 
expedite  parts  to  satisfy  MICAP 
conditions . 

DTE 

17 

1 

First  flight  [not  until]  Aug  1990. 

PM 

10 

6 

We  have  direct  access  to  field 
reports  and  use  this  data. 

Ik. 

DPML 

10 

5 

Classified  "off-line"  system  isn't 
as  flexible  as  it  could  be  in 
[performing]  trending. 

DPML 

3 

1 

See  response  to  question  la. 

OTHER 

14 

3 

Not  many  [to  choose  from]  yet. 

DTE 

1 

7 

Each  unit  provides  their  montnly 
maintenance  summary  for  our 
reviewers . 
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TABLE  IX  -  Continued 


VERBAL  RESPONSES  TO  QUESTION  1 


Ques . 
No. 

Duty 

Title 

#Yrs 

Acq. 

Rat 

ing 

Response 

11. 

DPML 

10 

2 

Performance  and  schedule  drove 
everything . 

DEN 

24 

6 

All  R&M  trade-offs  are  closely 
tracked . 

DPML 

3 

1 

See  response  to  question  la. 

DPML 

7  ' 

4 

See  response  to  question  li. 

DTE 

1 

5 

Dollar  shortfalls  appear  to  be  the 
greatest  constraint  to  the  program 
at  present.  We  are  continually 
reviewing  [the]  program  to 
maximize  [use]  of  resources. 

PM 

10 

1 

Not  required  at  this  point  in  the 
program. 

lm. 

DPML 

10 

3 

Too  little  data  was  required  and 
no  agreed  to  SPO-User-Contractor 
methods  caused  problems... 

DEN 

24 

2 

Too  much  data  and  not  enough 
information. 

DPML 

3 

1 

See  response  to  question  la. 

DTE 

1 

0 

Not  familiar. 

In. 

DPML 

10 

3 

Contractor  resists  change  from 
[his]  own  past  practices  and 
methods . 

DEN 

24 

6 

Gives  designers  good  insight. 

[They  are]  incorporated  in  the 
design  database. 

DPML 

3 

1 

See  response  to  question  la. 

DTE 

1 

6 

We  continually  review  our  lessons 
learned  from  previously  identified 
problems . 

DPML 

16 

4 

...are  more  useful  to  tell  you 
what  not  to  do  with  old  technology 
than  how  to  effectively  use  new 
technology. 

lo . 

DPML 

10 

1 

See  response  to  question  11. 

DEN 

24 

3 

Too  much  design  detail  emphasis  in 
in  a  Dem/Val  Program.  Needs  to  be 
more  focused  in  FSD . 

DPML 

3 

1 

See  response  to  question  la. 

DTE 

10 

0 

Not  familiar  with  this. 
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VERBAL  RESPONSES  TO  QUESTION  1 


Ques . 
No. 

Duty 

Title 

#Yrs 

Acq. 

Rat 

ing 

Response 

lo. 

PM 

10 

6 

The  earlier  the  better 
(ie,  Pre-RFP). 

Ip. 

DPML 

10 

3 

[Provides]  no  measurable 
contractor  motivation. 

DEN 

24 

6 

Contractors  have  to  be  told  (and 
were)  warranties  will  be  part  of 
the  FSD  contract. 

DPML 

3 

1 

See  response  to  question  la. 

DTE 

10 

0 

No  knowledge. 

DPML 

7 

5 

Again,  constrained  only  by  budget 
limitations. 

DPML 

8 

7 

Nine  requirements  have  $12  million 
in  incentives. 

iq. 

DEN 

24 

4 

Need  to  be  tailored  to  the  design. 

DPML 

3 

1 

See  response  to  question  la. 

PM 

10  + 

6 

If  [included]  in  specifications. 

DPML 

8 

1 

Don't  have  any,  other  than  R&M 
requirements  and  goals. 

DTE 

1 

0 

Not  familiar. 

PM 

10 

6 

Classical  measurements. 
Interpretation  is  [the]  key. 

lr . 

DPML 

10 

1 

Contractor  personnel  are  "status 
trackers"  rather  than  "design 
inf luencer s . " 

DEN 

24 

5 

...a  separate  R&M  personnel 
[section]  fosters  "it's  not  my 
job"  attitudes  by  the  designers. 

In  this  SPO,  they  are  integrated. 

DPML 

3 

1 

See  response  to  question  la. 

OTHER 

14 

5 

One  of  the  strongest. 

PM 

10 

NR 

Every  engineer  has  the  potential 
to  impact  R&M.  R&M  is  just  good 
design  engineering. 

Is . 

DPML 

10 

2 

Too  much  orientation  towards 
tracking  and  reporting  rather  than 
influencing  and  action. 

DEN 

24 

6 

All  [our]  contractors  have  strong 
policies  to  make  R&M  a  part  of 
[their]  engineering  organization. 

DPML 

3 

1 

See  response  to  question  la. 
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VERBAL  RESPONSES  TO  QUESTION  1 


Ques . 
No. 

Duty 

Title 

#Yrs 

Acq. 

Rat¬ 

ing 

Response 

Is . 

DTE 

10 

0 

No  knowledge. 

PM 

10  + 

6 

[Effective]  if  used. 

PM 

10 

5 

Got  to  watch  [them]  at  the  right 
points  in  the  process. 

It. 

DPML 

10 

5 

Points  out  proof  of  problems  not 
previously  recognized  or  admitted. 

DEN 

24 

5 

Only  a  small  part  of  Dem/Val. 

Will  receive  more  emphasis  in  FSD . 

DPML 

3 

1 

See  response  to  question  la. 

PM 

10  + 

7 

[Effective]  if  warranted. 

DPML 

8 

7 

[Effective.]  What  about  M-Demos? 

DTE 

1 

6 

All  new  systems  being  developed... 
must  demonstrate  that  they  meet  or 
exceed  reliability  goals  through 
testing . 

OTHER 

8 

6 

Very  effective  when  [you]  include 
all  the  lifetime  ( cradle-to-grave ) 
stresses . 

PM 

10 

6 

If  properly  used,  this  is  very 
effective . 

lu . 

DPML 

10 

6 

Weeds  out  bad  parts  before 
assembly . 

DEN 

24 

6 

[A]  major  emphasis  in  FSD. 

DPML 

3 

1 

See  response  to  question  la. 

DTE 

10 

1 

Not  applicable  at  this  point. 

OTHER 

14 

4 

Will  be  strong,  [tests]  still 
being  definitized. 

DPML 

16 

6 

Good  if  applied  correctly. 

PM 

10 

6 

See. response  to  question  It. 
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VERBAL  RESPONSES  TO  QUESTION  2 


Ques . 
No. 

Duty 

Title 

#Yr  s 
Acq. 

Rat¬ 

ing 

Response 

2a. 

DPML 

10 

6 

Forms  the  basis  for  decisions 
regarding  repair  levels,  spares, 
and  maintenance  training. 

DPML 

3 

1 

ILS  is  not  a  viable  program  [when 
dealing  with]  modifications. 

DTE 

10 

0 

[I  have]  no  knowledge  of  these 
efforts  on  this  program. 

DPML 

8 

6 

What  level?  O&I  or  D? 

DTE 

9 

0 

Outside  my  area  of  knowledge. 

DTE 

1 

7 

How  the  user  plans  to  support 
aircraft  is  key  to  developing  all 
the  ILS  elements. 

2b. 

DPML 

10 

5 

[The]  basis  for  the  number  of 
manpower  slots. 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response  to  question  2a. 

DTE 

9 

0 

See  response  to  question  2a. 

DTE 

1 

7 

R&M  requirements  have  a  major 
impact  on  the  number  of  people 
required. . . 

DEN 

28 

1 

[The]  user  estimates  manpower  level 
high  no  matter  how  stringent  the 

R&M  requirements  are! 

2c . 

DPML 

10 

7 

Dependent  on  R&M  predictions  (DPML 
doesn't  always  rely  on  contractor 
data,  and  has  to  make  his  own 
predictions ) . 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response  to  question  2a. 

DPML 

8 

6 

Preplanned  ILS. 

DTE 

9 

0 

See  response  to  question  2a. 

DTE 

1 

7 

Depth  of  spares  [is]  determined 
based  upon  expected  utilization 
and  failure  rates  of  components. 

2d. 

DPML 

10 

3 

An  outgrowth  of  maintenance 
planning . 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response  to  question  2a. 

DPML 

8 

7 

Support  equipment  by  capability. 

DTE 

9 

0 

See  response  to  question  2a. 

TABLE  X  -  Continued 


VERBAL  RESPONSES  TO  QUESTION  2 


Ques . 
No. 

Duty 

Title 

#Yr  s 
Acq. 

Rat¬ 

ing 

Response 

2d. 

DTE 

1 

7 

High  reliability  [means]  a 
candidate  for  two-level  repair 
[and]  SE  at  the  intermediate  level 
is  not  needed. 

2e . 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response. to  question  2a. 

DPML 

8 

7 

Generation  from  LSAR . 

DTE 

9 

0 

See  response  to  question  2a. 

DTE 

1 

7 

The  better  the  tech  data,  the 
better  the  supportability . 

2f . 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response  to  question  2a. 

DPML 

8 

5 

Less  than  5%  of  the  program,  but  a 
big  schedule  driver. 

DTE 

9 

0 

See  response  to  question  2a. 

DTE 

1 

7 

Our  CDS  system  tracks  all 
maintenance  actions  and  provides 
the  depot  with  needed  R&M  data. 

2g. 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response  to  question  2a. 

DPML 

8 

6 

Contracted  to  another  SPO. 

DTE 

9 

0 

See  response  to  question  2a. 

DTE 

1 

7 

...MC  rates  continue  to  improve... 
as  the  learning  curve  goes  up. 

You  can  only  take  advantage  of  the 
full  capability  of  the  systems  if 
you  have  trained  people. 

2h . 

DPML 

10 

7 

[Having  significant  influence]  is 
what  design  interface  is  all  about. 

DEN 

24 

7 

The  emphasis  that  R&M  is  the 
designer's  job  and  not  logistics' 
or  other  disciplines'  [has]  the 
single  biggest  impact  in  assuring 
R&M  is  built  in,  not  added  on. 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response  to  question  2a. 

DTE 

9 

0 

See  response  to  question  2a. 

DTE 

1 

7 

Ease  of  maintenance .. .must  be 
considered  early  in  [the]  design 
stage . 

TABLE  X  -  Continued 
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VERBAL  RESPONSES  TO  QUESTION  2 


Ques . 
No. 

Duty 

Title 

#Yr  s 
Acq . 

Rat¬ 

ing 

Response 

PM 

10 

2 

[It]  should  be  a  7. 

2i. 

DEN 

24 

3 

Too  early  yet  [for  this  to  be  an 
issue ] . 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response  to  question  2a. 

DTE 

9 

0 

See  response  to  question  2a. 

DTE 

1 

5 

R&M  suffers  severely  if  facilities 
are  lacking. 

2j. 

DPML 

3 

1 

See  response  to  question  2a. 

DTE 

10 

0 

See  response  to  question  2a. 

DPML 

8 

4 

We  have  commercial  items,  their 
packaging  may  be  different. 

DTE 

9 

0 

See  response  to  question  2a. 

DTE 

1 

7 

Even  the  most  reliable  systems  are 
degraded  if  the  LRUs  are  not 
properly  handled.  This  is  a 
continual  problem  throughout  the 

Air  Force,  regardless  of  the 
program. 

97 


TABLE  XI 


VERBAL  RESPONSES  TO  QUESTION  6 


Duty 

Title 

#Yrs 

Acq. 

Ans . 

Response 

DEN 

30 

NO 

We  need  to  train  R&M  fundamentals  (ie, 
mathematics  and  how  to  prepare  RFPs  and 
specifications)  and  to  provide  OJT  for 
the  application  of  fundamentals. 

DTE 

8 

YES 

With  [the]  two-level  maintenance  concept 
...ours  should  be  a  very  easy  system  to 
field.  [The]  level  of  support  appears 
to  be  adequate. 

DEN 

17 

NO 

Too  many  new  initiatives. . .and  not 
enough,  people  to  manage  or  even  comment 
on  them  ( ie ,  AVIP ) . 

PM 

20 

NO 

Never  was,  never  will  bel 

DEN 

28 

NO 

The  only  thing  we  do  is  document  the 
lack  of  R&M  i  We  do  not  put  proper 
emphasis  up  front  via  manufacturing 
functional  expertise. 

DPML 

3 

NO 

R&M  in  this  SPO  belongs  to  ASD 
Engineering.  They  are  coming  on  line, 
but  slower  than  I  would  like. 

DPML 

10 

NO 

[I  have:]  1.  a  civil  service  projects 
manager  with  no  R&M  experience,  2.  a 

DPML  staff  engineer  with  no  previous 

R&M  experience,  and  3.  an  EN  staff 
engineer  at  [the]  worker  level  without 
strong  management  support. 

DEN 

18 

NO 

While  the  logistics  group  has  no  people 
to  support  R&M,  there  is  only  one  R&M 
engineer  assigned.  This  engineer  only 
has  time  to  track,  and  very  little  time 
to  work  R&M  issues. 

DEN 

24 

YES 

If  the  engineering  staff  recognizes  that 
it  is  their  responsibility  to  make  R&M 
happen,  only  a  modest  increase  [in] 
overseers  who  understand  the  "language" 
is  required.  A  gross  expansion  will 
reinforce  the  "it's  not  my  job" 
attitude. 

OTHER 

3 

NO 

More  people  are  needed  to  monitor  R&M- 
associated  testing  for  quick  reporting 
of  results. 

OTHER 

7 

NO 

All  R&M  personnel  support  is  from  the 
home  office.  To  do  an  adequate  job,  a 
full-time  individual  is  required  [on 
station  3 . 

TABLE  XI  -  Continued 


VERBAL  RESPONSES  TO  QUESTION  6 


Duty 

Title 

#Yrs 
Acq . 

Ans . 

Response 

DEN 

8 

NO 

The  current  opinion  of  the  R&M  career 
area... has  not  drawn  the  best  people. 

DPML 

3 

YES 

A  pool  exists,  however  all  have  other 
areas  that  their  attention  besides  R&M. 

DTE 

2 

YES 

We  have  a  vast  amount  of  operational 
experience,  as  well  as  competent 
logisticians. 

PM 

27 

NO 

[I]  have  personnel  shortages  across  the 
board . 

DPML 

1 

NO 

Under  the  ASD  reorganization,  Logistics 
Directorate  was  not  allowed  a  staff. 

All  the  specialty  "experts"  were 
absorbed  into  the  expanded  program 
directorates.  Therefore,  we  lost  access 
to  experienced  R&M  personnel.  From  a 
logistics  viewpoint,  the  reorganization 
stinks . 

OTHER 

14 

NO 

We  have  one — to  share  across  three 
major  efforts.  Could  use  more — at  least 
one  per  major  effort. 

DPML 

8 

NO 

Not  enough  to  cover  vendor  PDRs  and 

CDRs .  Can  only  manage  to  [the]  prime 
[contractor]  level. 

OTHER 

36 

NO 

Are  there  any  [at  all]?  Answer — NO! 

DTE 

9 

YES 

Combination  of  engineering  and  projects. 
The  engineers  [have]  an  R&M  background, 
the  project  manager  is  self-taught. 

DTE 

1 

YES 

The  majority  of  my  personnel  have 
previous  aircraft  maintenance 
experience.  The  DPML  has  a  large  cadre 
of  experienced  R&M  personnel.  This  all 
adds  up  to  a  good  experience  mix  and 
serves  the  program  well. 

OTHER 

24 

NO 

I  have  one  R&M  engineer  who  works  with 
approximately  15  other  engineers  on  a 
$4  billion  production  program  with  a 
large  number  of  R&M  changes  being 
incorporated...  One  person  is  not 
enough  to  track  the  R&M  data  for  this 
size  program,  given  its  dynamic  nature. 

DEN 

28 

NO 

[The]  quality  of  personnel  has  to  be 
improved . 
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VERBAL  RESPONSES  TO  QUESTION  6 


Duty  #Yrs 

Title  Acq.  Ans.  Response 


OTHER  8  NO 


DEN  29  NO 


DPML 

16 

NO 

PM 

10 

YES 

The  majority  of  people  who  work  for  me 
are  inexperienced  and  undertrained. 
[They]  have  to  be  trained  through  OJT 
and  formal  R&M  courses...  Their 
technical  work  has  to  be  continually 
monitored.  This  process  takes  about  two 
or  three  years...  Also,  it  is  even 
tougher  when  a  manager  is  dealing  with 
military  [personnel  who  are]  only  with 
the  organization  two  to  four  years. 

Only  one  R&M  manager  assigned.  On  a 
major  program  [it  is]  impossible  to  keep 
up  with  prime  and  major  subcontractors 
even  though  [personnel  are]  always  TDY . 
Excessive  TDY  detracts  from  adequate 
planning  and  assessment,  and 
communication  with  [the]  Chief  Engineer 
and  management...  [This]  has  created  a 
health  problem  requiring  some  time  off 
that  further  aggravates  [the]  problem. 
There  is  only  one  in  the  engineering 
organization . 

I've  got  all  the  engineers  I  need. 
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VERBAL  RESPONSES  TO  QUESTION  8 


Duty 

Title 

#Yrs 

Acq. 

Ans . 

Response 

DEN 

17 

4 

Too  difficult  to  implement  due  to  lack 
of  experienced  people... 

DEN 

28 

2 

No  training/insight  on  how  to  do  it. 

DPML 

3 

5 

Current  engines  are  reliable  and 
maintainable.  Future  engines  will  be 
[even]  more  so. 

DPML 

10 

3 

[This]  program  began  before  R&M  2000. 

But  visibility  of  efforts  has  helped  to 
motivate  [the  prime]  contractor  and  subs. 

DEN 

18 

2 

Some  policy  letters  have  been  received 
and  implemented. 

DEN 

24 

7 

R&M  has  one  of  the  highest  priorities  in 
the  program,  right  after  affordability 
and  weight.  Only  by  showing  the 
continued  high  level  of  Air  Force 
commitment  to  R&M  can  a  SPO  keep  the 
contractors  focused  on  this  critical 
part  of  weapon  system  design. 

OTHER 

3 

7 

We  have  made  the  contractor  believe  in 
the  R&M  2000  Program.  He  has  seen 
significant  [improvement]  in  his  product 
because  of  it. 

DTE 

9 

5 

We  buy  primarily  commercial,  off-the- 
shelf  aircraft. 

OTHER 

7 

5 

Program  [management]  is  handled  from  the 
home  office,  not  at  [the]  location  where 
[the]  work  is  being  done. 

DEN 

8 

6 

It  has  had  the  proper  effect.  There  is 
an  awareness  of  the  need  for  [R&M] 
improvements.  At  the  current  point  of 
production,  there  is  only  so  much  that 
can  be  done  to  improve  either. 

DPML 

3 

2 

Not  a  great  need. 

DTE 

2 

6 

...R&M  is  effectively  administered,  but 
it  is  not  an  everyday  topic  of 
conversation  in  my  office.  Need  more, 
feedback  from  the  people  who  track  this 
effort . 

PM 

27 

5 

Effectiveness  really  depends  on  top 
management  emphasis  and  support.  R&M 

2000  would  [just]  be  another  "-ility" 
without  this.  I  believe  high  level 
emphasis  is  engrained  in  all  of  today's 
major  programs. 
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VERBAL  RESPONSES  TO  QUESTION  8 


Duty 

Title 

#Yr  s 
Acq. 

Ans . 

Response 

DPML 

1 

2 

Not  enough  emphasis  and  support  from  the 
SPO. 

PM 

10 

7 

[The]  big  push  on  R&M  2000  clearly  had 
an  impact  on  our  contractors.  They 
manifest  continuing  commitment  to  a  more 
reliable  maintenance  design. 

DTE 

10 

0 

Not  familiar  [with  the  R&M  2000  Program]. 

PM 

10+ 

7 

[Ours  is]  considered  a  model  program. 

OTHER 

4 

2 

We  speak  to  R&M  2000  but  show  little 
progress  towards  achieving  its 
objectives.  Our  efforts  are  primarily 
focused  on  requiring  adequate 
reliability  measures  to  be  contracted 
for  in  ILS  contracts  (ie,  established 

R&M  values  as  targets  to  be  met  and 
financial  penalties  [levied]  if  not). 

OTHER 

36 

1 

[I]  don't  have  people  to  work  any  such 
program. 

OTHER 

24 

6 

Dr.  Halpin  [ASD  Asst  for  Product 
Assurance]  has  taken  a  personal  interest 
over  a  number  of  years.  This  has  placed 
emphasis  on  the  contractor  to  implement 
the  goals  of  R&M  2000  even  when  it  is 
not  on  contract  or  written. 

DEN 

28 

4 

Concerned  personnel  plus  definitive 
design  requirements,  test  programs,  and 
dollar  incentives  make  an  R&M  program 
effective . 

OTHER 

8 

6 

R&M  2000  is  providing  R&M  managers  with 
the  power  to  implement  the  necessary  R&M 
qualitative  and  quantitative 
requirements  on  contract  without  having 
the  program  managers  omitting  them. 

DEN 

29 

6 

R&M  2000  emphasis,  along  with  program 
reliability  problems,  has  driven 
aggressive  reliability  program  changes 
in  the  areas  of  testing,  growth  curve 
clarification,  and  tracking  and 
definition  of  a  coherent  [R&M]  program 
philosophy. 
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VERBAL  RESPONSES  TO  QUESTION  9 

Duty  #Yrs 

Title  Acq.  Ans.  Response 

DEN  30  YES  Train  managers  on  how  and  why  we  should 

provide  good  management  of  R&M. . . 

Don't  leave  [it]  up  to  the  technical 
organization.  Program  priorities  are 
set  by  program  managers. 

DEN  17  YES  More  talented  people.  Too  many  R&M 

people... are  cast-offs  from  the 
engineering  specialties — people  who 
can't  make  it.  They  are  too 
bureaucratic  and  not  innovative  or 
thinkers.  Also,  need  more  emphasis 
from  program  management. 

PM  20  YES  People  of  all  backgrounds  need  to  know 

the  basics. 

DEN  28  YES  Closer  coordination  between  design  and 

manufacturing  functions... 

Manufacturing  needs  to  get  more 
involved  sooner  both  at  ASD,  and 
perhaps  more  importantly,  at  the 
contractor's  plant. 

DPML  3  YES  Better  up-front  requirements. 

DPML  10  YES  R&M  are  functions  of  design,  and  Air 

Force  management  tends  to  set  [them] 
aside  as  a  separate  function.  We  need 
to  get  all  engineers  involved!  It's 
exactly  like  quality — you  have  to  build 
it  in,  not  inspect  it  in! 

DEN  18  YES  Education  and  training. 

DPML  10  YES  R&M  initiatives  integrated  with  design. 

DEN  24  YES  Stop  expanding  R&M  as  a  thing  separate 

from  the  basic  process.  More  resources 
devoted  by  the  Air  Force  to  track, 
audit,  direct,  etc.,  specific  R&M 
[issues]  causes  the  contractor  to 
develop  an  "it's  not  my  job" 
[attitude]...  The  corporate  Air  Force 
needs  to  continually  emphasize  [R&M]  as 
part  of  the  design  process  and  not  just 
indulge  in  the  "empire  building" 
associated  with  a  new  thrust. 

OTHER  3  YES  Training  on  available  computer  analysis 

programs . 
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TABLE  XIII  -  Continued 


VERBAL  RESPONSES  TO  QUESTION  9 


Duty 

Title 

#Yrs 

Acq. 

Ans . 

Response 

DTE 

9 

YES 

Standardized  test  parameters  and 
established  RM&A  criteria  vice 
different  [criteria]  for  each  user. 

DEN 

8 

YES 

...education  for  senior  level 
management.  Without  senior  level 
management  commitment,  R&M  is  left  by 
the  wayside  when  performance  is  in 
jeopardy. 

DPML 

3 

NO 

People  assigned  to  the  R&M  program 
would  be  better  put  to  use  in  the  SPOs. 
Let  the  program  managers  work  R&M  as 
required.  If  they  need  help,  provide 
training.  We  must  stop  this  stupid 
layered  management. 

DTE 

2 

YES 

Communicate  more  specifics  on  how  to 
apply  R&M  to  particular  programs,  and 
the  responsibilities  of  each 
organization. 

PM 

27 

YES 

Adequate  personnel  manning. 

DPML 

1 

YES 

Logistics  and  R&M  organizations  need  to 
be  given  more  autonomy  and  influence  in 
ASD.  Right  now,  it  is  no  more  than  lip 
service. 

PM 

10 

NO 

Too  many  initiatives  all  at  once. 
Everyone  wants  to  jump  on  the  R&M  2000 
bandwagon.  Let's  take  some  time  to  see 
how  the  next  generation  of  weapon 
systems  has  benefitted  before  our  next 
round  of  initiatives. 

DPML 

7 

YES 

[Keep  the]  budget  constant. 

PM 

10  + 

YES 

More  extensive  use  of  warranties,  and 
associated  realistic  demonstrations  and 
[an]  effective  warranty  tracking  system 
(other  than  66-1  data). 

DPML 

8 

YES 

Keep  the  [ASD  internal  affairs] 
newsletter  coming.  [Plus]  more 
emphasis  on  data  collection  and 
analysis  systems. 

OTHER 

4 

YES 

Education  of  what  R&M  is  and  how  it 
effects  life  cycle  costs... for  program 
management  personnel. 

OTHER 

36 

YES 

Spend  more  time  with  reasonably 
tailored  requi rements--less  time  trying 
to  cram  all  programs  into  a  mold. 
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TABLE  XIII  -  Continued 


VERBAL  RESPONSES  TO  QUESTION  9 


Duty 

Title 

#Yrs 

Acq. 

Ans . 

Response 

OTHER 

24 

YES 

Contractor  high  level  management  has  to 
take  on  the  commitment  to  provide 
reliable  and  maintainable  hardware  at 
the  beginning  of  a  program. . .Air  Force 
management  [has]  to  get  this  message  to 
industry .. .R&M  2000  has  helped  to 
accomplish  this  to  a  degree... 

DEN 

28 

YES 

Have  more  acquisition  personnel  with 
operational  experience. 

OTHER 

8 

YES 

[More  emphasis  is  needed]  in  the 
detailed  implementation  of  ASD's 
integrity  program,  and  RFP  streamlining 
policies  and  guidelines.  Training  is 
needed  at  all  levels  of  engineering  and 
management. 

DTE 

17 

YES 

New  programs  don't  benefit  from  other 

ASD  programs--lessons  learned  are 
"eyewash. * 

DEN 

29 

YES 

...more  personnel  trained  in  R&M 
program  structure,  trade-offs,  theory 
and  mathematics ...  training  [for] 
functional  engineers  to  familiarize 
[them]  with  R&M  program  goals,  impacts, 
importance,  and  how  to  achieve  high 
reliability  in  the  various  technical 
disciplines . 

DPML 

16 

YES 

More  trained  R&M  engineers,  career 
progression  in  the  field,  less  high 
level  "gee-whiz"  programs  so  the 
workers  can  concentrate  on . . . detai Is . 

PM 

10 

YES 

Educate  R&M  managers  about  systems 
acquisition  and  program  management.  We 
don't  need  more  policies  and  special 
interest  organizations  that  are  well 
intentioned  but  naive  about  the 
acquisition  process.  We  need  doers — 
not  watchers  and  overseers. 

PM 

10 

YES 

[We  need  more  emphasis  on  R&M]  at  the 
engineering/manufacturing  interface . 

PM 

15 

YES 

Maintainability. 

APPENDIX  C 

ADDITIONAL  TABLES  AND  FIGURES  FROM  ANALYSIS 


OF  QUESTIONS  1,  2,  AND  8 


TABLE  XIV 


SURVEY  DATA  -  QUESTION  1 


la . 

SOW 

Ik. 

R&M  MISS 

lb. 

SON 

11. 

R&M  Program  Constraints  and 

lc . 

soc 

Tradeoffs 

Id. 

RCM 

lm. 

R&M  Data  Requirements 

le. 

R&M  Management  Plan 

In . 

R&M  Lessons  Learned 

If. 

MAJCOM  R&M  Plan 

lo. 

Timing  of  R&M  Activities 

ig. 

Instructions  to 

Ip. 

Incentives  and  Warranties 

Offerers 

iq. 

R&M  Measures  of  Merit 

lh. 

SPO  R&M  Organization 

lr . 

R&M  Personnel  Management 

li. 

New  Applications  of 

Is . 

Documented  Contractor  R&M 

R&M  Technology 

Policies  and  Practices 

lj. 

Collection  and  Use 

It. 

Reliability  Demonstrations 

of  Field  R&M  Data 

lu. 

ESS 

Ques# 

NR 

0 

1 

Responses 

2  3  4  5 

6 

7 

MEAN 

Z- 

VALUE 

REJECT 

NULL? 

la. 

1 

1 

1 

0 

1 

10 

14 

13 

5 

5.04 

15.635 

YES 

lb. 

1 

3 

1 

2 

6 

14 

11 

7 

1 

4.07 

9.458 

YES 

lc . 

1 

3 

0 

3 

8 

13 

11 

5 

2 

4.02 

9.348 

YES 

Id. 

3 

4 

4 

2 

7 

12 

6 

7 

1 

3.63 

6.472 

YES 

le . 

2 

3 

0 

2 

4 

12 

12 

9 

2 

4.36 

10.370 

YES 

If. 

1 

4 

8 

4 

7 

14 

6 

1 

1 

3.02 

4.873 

YES 

ig. 

0 

4 

5 

3 

3 

8 

8 

12 

3 

4.02 

7.184 

YES 

lh. 

1 

0 

2 

2 

3 

8 

10 

13 

7 

4.98 

13.634 

YES 

li. 

2 

1 

4 

4 

5 

9 

7 

12 

2 

4.18 

8.798 

YES 

Ij. 

2 

1 

7 

1 

4 

5 

9 

9 

8 

4.45 

8.426 

YES 

Ik. 

2 

1 

3 

1 

7 

10 

13 

8- 

1 

4.23 

10.480 

YES 

11. 

3 

2 

4 

3 

2 

13 

9 

10 

0 

4.02 

8.441 

YES 

lm. 

1 

3 

3 

4 

5 

10 

12 

6 

2 

3.91 

7.861 

YES 

In . 

1 

1 

3 

1 

5 

11 

10 

12 

2 

4.44 

10.986 

YES 

lo . 

2 

3 

4 

1 

8 

6 

10 

8 

4 

4.09 

7.727 

YES 

Ip. 

1 

2 

5 

2 

2 

3 

11 

13 

7 

4.64 

9.240 

YES 

lq. 

0 

5 

4 

3 

3 

8 

13 

7 

3 

3.89 

6.916 

YES 

lr . 

3 

2 

5 

3 

7 

5 

13 

8 

0 

3.84 

7.500 

YES 

Is . 

2 

4 

3 

2 

4 

10 

11 

9 

1 

3.98 

7.684 

YES 

It. 

1 

1 

3 

2 

2 

6 

11 

13 

7 

4.87 

11.509 

YES 

lu . 

0 

1 

3 

0 

1 

3 

14 

20 

4 

5.13 

14.328 

YES 
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PLANNING  TOOLS 


FREQUENCY 


QUESTION  NUMBER 


Figure  17:  Question  1  Frequency  Distribution  I 


1H  1M 

QUESTION  NUMBER 


Figure  18:  Question  1  Frequency  Distribution  II 


QUESTION  NUMBER 


li  da 


rating 

Is  M2  *  d!  s 


MOTE:  **50  Response"  Dete  Met  Included 


Figure  19:  Question  1  Frequency  Distribution  III 


CONTROLLING  TOOLS 

FREQUENCY 
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Figure  20:  Question  1  Frequency  Distribution  IV 


I  MANAGEMENT  TOOLS  OVERALL 

! 

i  FREQUENCY 


RATING 

o  Si  d!  2  Bi  HsU  Cs  3B«  ZD  r  j 


NOTE;  *?fo  Response  '  Date  Mot  Included 


Figure  21:  Question  1  Frequency  Distribution  V 
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TABLE  XV 


SURVEY  DATA  -  QUESTIONS  2  AND  8 


2a . 

Maintenance  Planning  (MXP) 

2b. 

Manpower  and  Personnel  (M&P) 

2c . 

Supply  Support  (SS) 

2d. 

Support  Equipment  (SE) 

2e . 

Technical  Data  (TD) 

2f . 

Computer  Resources  (CR) 

2g. 

Training  (TNG) 

2h . 

Design  Interface  (DI) 

2i . 

Facilities  (FAC) 

2  j. 

Packaging,  Handling,  Storage 
and  Transportation  (PHS&T) 

8. 

The  Effectiveness  of  R&M  2000 

Ques# 

NR 

0 

1 

Responses 

2  3  4  5 

6 

7 

MEAN 

2- 

VALUE 

REJECT 

NULL? 

2a. 

1 

4 

1 

2 

1 

5 

15 

13 

4 

4.64 

9.987 

YES 

2b. 

1 

5 

2 

2 

3 

14 

11 

5 

3 

3.93 

7.522 

YES 

2c . 

1 

5 

1 

2 

4 

9 

10 

9 

5 

4.27 

8.150 

YES 

2d. 

1 

4 

2 

1 

2 

6 

10 

14 

6 

4.67 

9.253 

YES 

2e. 

1 

4 

4 

1 

4 

10 

11 

9 

2 

4.02 

7.602 

YES 

2f . 

1 

5 

5 

1 

4 

13 

9 

7 

1 

3.67 

6.466 

YES 

2g. 

1 

5 

4 

1 

5 

11 

10 

8 

1 

3.78 

6.920 

YES 

2h . 

1 

4 

3 

1 

3 

5 

11 

10 

8 

4.56 

8.675 

YES 

2i . 

1 

5 

3 

0 

6 

13 

13 

5 

0 

3.73 

7.303 

YES 

2j. 

1 

5 

3 

1 

3 

18 

10 

4 

1 

3.71 

7.160 

YES 

8. 

3 

1 

2 

6 

1 

8 

n 

4.47 

9.983 

'  ~  1 
YES 

ILS  ELEMENTS  OVERALL 


ILS  ELEMENT 


NOTE:  "Ho  RasponM*  D«U  Mot  Included 


Figure  22:  Question  2  Frequency  Distribution 


APPENDIX  D 


LIST  OF  ABBREVIATIONS 


Ac  ft 

AFALC 

AFALD 

AFIT 

AFMPC 

AFSC 

ALAC 

Ar  If  t 

ASD 

ATF 

AVIP 

BCM 

CDR 

CDS 

CERM 

CR 

Dem/Val 

DEN 

DI 

DOD 

DODD 

DPML 

DSB 

DSMC 

DTE 

EN 

Eng 

ESS 

FAC 

FSD 

HQ 

Hyp 

IDA 

ILS 

LOG 

LRU 

LSAR 

M&P 

MAJCOM 

MC 

M-Demo 

MICAP' 

MIS 

MXP 


Aircraft 

Air  Force  Acquisition  Logistics  Center 
Air  Force  Acquisition  Logistics  Division 
Air  Force  Institute  of  Technology 
Air  Force  Military  Personnel  Center 
Air  Force  Systems  Command 
Airlift  Aircraft. 

Airlift 

Aeronautical  Systems  Division 
Advanced  Tactical  Fighter 
[ASD]  Avionics  Integrity  Program 
Baseline  Correlation  Matrix 
Critical  Design  Review 
[ F— 1 6 ]  Central  Data  System 
Center  of  Excellence  for  Reliability  and 
Maintainability 
Computer  Resources 

Demonstration/Validation  [acquisition  cycle  phase] 

Deputy  for  Engineering 

Design  Interface 

Department  of  Defense 

Department  of  Defense  Directive 

Deputy  Program  Manager  for  Logistics 

Defense  Science  Board 

Defense  Systems  Management  College 

Deputy  for  Test  and  Evaluation 

Engineering 

Engine 

Environmental  Stress.  Screening 
Facilities 

Full  Scale  Development  [acquisition  cycle  phase] 

Headquarters 

Hypothesis 

Institute  for  Defense  Analysis 
Integrated  Logistics  Support 
Logistics 

Line  Replaceable  Unit 

Logistics  Support  Analysis  Report 

Manpower  and  Personnel 

Major  Command 

Mission  Capable 

Maintainability  Demonstration 

Mission  Capable 

Management  Information  System 

Maintenance  Planning 
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Msl 

NR 

OJT 

OSD 

PDR 

PHS&T 

PM 

R&M 

RCM 

RFP 

RM&A 

SE 

SOC 

SON 

SOW 

Spec(s ) 

SPO 

SS 

Std  Dev 

STRATAC 

STRATMSL 

TACAC 

TACENG 

TACMSL 

T&E 

TAF 

TD 

TDY 

TNG 


Missile 

No  Response 

On-the-Job  Training 

Office  of  the  Secretary  of  Defense 

Production  Design  Review 

Packaging,  Handling,  Storage  and  Transportation 
Program  Manager 

Reliability  and  Maintainability 
Requirements  Correlation  Matrix 
Request(s)  for  Proposals 

Reliability,  Maintainability,  and  Availability 
Support  Equipment 

Statement  of  Operational  Capability 
Statement  of  Need 
Statement  of  Work  . 

Specif ication( s ) 

System  Program  Office 
Supply  Support 
Standard  Deviation 
Strategic  Aircraft 
Strategic  Missile 
Tactical  Aircraft 
Tactical  Engine 
Tactical  Missile 
Test  and  Evaluation 
Tactical  Air  Forces 
Technical  Data 

Temporary  Duty  [away  from  home  station] 

Training 
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